-- Logs begin at Thu 2022-06-16 15:17:44 UTC, end at Thu 2022-06-16 16:45:17 UTC. -- Jun 16 15:57:59.775296 dv-cinder-837354-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 15:57:59.784595 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 15:57:59 2022] *** Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: nodename: dv-cinder-837354-2-iscsi Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: machine: x86_64 Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: clock source: unix Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: pcre jit disabled Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: detected number of CPU cores: 4 Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: current working directory: / Jun 16 15:57:59.785685 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: detected binary path: /usr/bin/uwsgi-core Jun 16 15:57:59.786376 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: your processes number limit is 47680 Jun 16 15:57:59.786376 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: your memory page size is 4096 bytes Jun 16 15:57:59.786376 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: detected max file descriptor number: 2048 Jun 16 15:57:59.786376 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: lock engine: pthread robust mutexes Jun 16 15:57:59.786376 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: thunder lock: enabled Jun 16 15:57:59.786376 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 15:57:59.786376 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 15:57:59.832024 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: Python main interpreter initialized at 0x55acf92439c0 Jun 16 15:57:59.832024 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: python threads support enabled Jun 16 15:57:59.832024 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: your server socket listen backlog is limited to 100 connections Jun 16 15:57:59.832024 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: your mercy for graceful operations on workers is 90 seconds Jun 16 15:57:59.832487 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 15:57:59.832487 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: *** Operational MODE: preforking *** Jun 16 15:57:59.832754 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: *** uWSGI is running in multiple interpreter mode *** Jun 16 15:57:59.832754 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: spawned uWSGI master process (pid: 80388) Jun 16 15:57:59.832950 dv-cinder-837354-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 15:57:59.834175 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: spawned uWSGI worker 1 (pid: 80395, cores: 1) Jun 16 15:57:59.836060 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: spawned uWSGI worker 2 (pid: 80396, cores: 1) Jun 16 15:57:59.836060 dv-cinder-837354-2-iscsi devstack@keystone.service[80388]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 15:58:05.340879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:58:05.340879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes Jun 16 15:58:05.341678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:58:05.341678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes Jun 16 15:58:05.344425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:58:05.344425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: from cryptography.utils import int_from_bytes Jun 16 15:58:05.344883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 15:58:05.344883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: from cryptography.utils import int_from_bytes Jun 16 15:58:05.392444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 15:58:05.393399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 15:58:05.393600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 15:58:05.393825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] command line args: [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 15:58:05.394476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 15:58:05.394622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 15:58:05.394704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] command line args: [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 15:58:05.394803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ================================================================================ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 15:58:05.394957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 15:58:05.395064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.395151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ================================================================================ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 15:58:05.395315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_dir = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.395402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] admin_token = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.395505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.395632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_dir = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.395711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_source = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.395827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.395915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] debug = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_source = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] debug = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.396975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_config_append = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_config_append = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.397958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_dir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_dir = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.398994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.399906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.400973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.401928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] publish_errors = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] publish_errors = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402852 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.402972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] standard_threads = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.403994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] standard_threads = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_journal = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_json = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.404974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_journal = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.405129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_stderr = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.405312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_json = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.405469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_syslog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.405614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_stderr = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.405802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.405930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_syslog = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.406031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.406031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.406258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 15:58:05.406410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.406535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.406660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.406782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.406899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.406899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.407923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.408930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.409056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.409177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.409292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.409292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.409496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.410976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.411969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.412932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.413893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.414959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.415995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.416994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.417944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.418973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.419987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420109 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.420954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.421939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.422901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423006 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.423974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424109 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.424947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.425905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.426976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427078 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.427956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.428897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.429887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.external = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.430982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.external = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431425 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.password = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.token = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.431878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.password = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.token = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.432929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433178 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.433975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.434915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.435864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.436009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.436135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.436580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.436773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.436921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.437912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.438884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.439918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.440037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.440156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.440275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.440393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.440519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.440637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.441252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.441376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.441494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.441611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.441880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.442959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.443924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.444059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.444175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.444292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.444410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.444527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.444653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.444771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.445975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.446930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.447891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.448963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.449998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.450118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.450237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.450353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.450591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.450717 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.450845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.450964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.451935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.452903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.453890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.454978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.455924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.456042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.456160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.456277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.456394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.456560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.457184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.457304 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.457421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.457539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.457657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.457840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.457964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458322 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.458905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.459928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.460973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.461925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.462908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.463992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464761 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.464962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.465950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.466937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467831 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.467911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.468983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.469068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.469234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.469465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.469547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.469710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.469847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.469990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.470135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.470257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.470395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.470531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.470651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.470769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.470884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.471956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.472936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.473924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.474973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.475917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.476033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.476158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.476276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.476393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.476510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.479599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.479738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.479880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.480967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.481967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.482909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.483979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.484923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.485040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.485157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 15:58:05.485291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.485291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.485291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.485291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 15:58:05.485291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 15:58:05.600555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.600784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.600876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.600955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.601031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.601107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.609515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.609620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.638178 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.638631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.638716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.638716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.692663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.692891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.741995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.742230 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.743071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.743310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.808954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.809180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.809407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.809489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.809632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.809736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.809881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.809972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.810098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.810195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.810315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.810532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.834290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.834495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.834607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.834730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.834813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.834955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.835815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.846708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.846980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.859867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.860956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.861039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.861170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.861261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 15:58:05.878853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.879063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:58:05.879266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.879402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.879502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:58:05.879607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:58:05.879798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.879880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.879999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 15:58:05.880116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.880266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.880370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.880526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.880607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.880783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.881039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.891623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.891897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.892008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.892161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.892286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.892437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.892557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.892842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.901143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.901581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.904412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.904693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.904869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.904973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.905149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.905242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.905435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.905516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.905730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.905815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.906895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.907077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.907157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.907347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.907613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.934358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.934678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.935955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.936259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.936365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.936554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.941500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.941865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.941949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.942183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.942277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.942533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.949684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.949940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.950050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.950329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.954367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.954648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.954801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.954928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.955084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.955202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.955365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.955469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.955647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.955727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.955922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.956004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.956197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.956277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.956468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.957254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.975579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.975853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.976040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.976317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.980666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.981258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.984278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.984671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.984898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.985293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.990220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.990830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.993792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.994450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:05.997431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.997740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.998028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.998109 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.998308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.998420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.998585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.998705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.999075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.999172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.999304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:05.999579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.018296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.018579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.018662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.018841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.018940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.019615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.019718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.019902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.027563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.028042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.029265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.029540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.029776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.030073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.034340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.034622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.034706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.034983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.035102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.035261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.035351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.035532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.035614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.035814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.048673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.048949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.049034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.049227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.049309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.049502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.049594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.049805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.049911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.050179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.050261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.051000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.051104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.051288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.068778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.069084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.072314 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.072871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.075938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.076175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.077671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.077935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.079328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.079596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.079680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.079874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.079956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.080154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.087605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.087905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.089191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.089502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.092407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.092726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.092821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.093010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.093094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.093339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.093423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.093658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.102550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.102842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.103100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.103198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.103386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.103665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.110903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.111178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.111454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.111660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 15:58:06.111746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.111940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.112193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.112276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.112478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.112664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.112748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.113464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.113651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.113764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.113956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.114038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.114232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.114369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.114517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.114599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.114788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.114868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.115059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.115142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.115326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.115405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.115590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.115864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 15:58:06.153922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55acf92439c0 pid: 80395 (default app) Jun 16 15:58:06.154288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55acf92439c0 pid: 80396 (default app) Jun 16 15:58:06.157409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9f82c-cd44-4dcb-bf56-3ee6a03c59b9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:06.157704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9f82c-cd44-4dcb-bf56-3ee6a03c59b9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:06.158007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9f82c-cd44-4dcb-bf56-3ee6a03c59b9 None None] PATH_INFO: `/v3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:06.162868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Thu Jun 16 15:58:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:14.002545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d5de-d4b8-42dc-9f08-b63268c135c8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.003125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d5de-d4b8-42dc-9f08-b63268c135c8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.003125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1d5de-d4b8-42dc-9f08-b63268c135c8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.007953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1/2] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:14.013374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.013830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.013962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.083285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: INFO dbcounter [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] Registered counter for database keystone Jun 16 15:58:14.105025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG oslo_db.sqlalchemy.engines [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 15:58:14.111390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writer thread running {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 15:58:14.290743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80395) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 15:58:14.291079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80395) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 15:58:14.346572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:14.348233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:14.475165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 2/3] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 15:58:14.617976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1d5de-d4b8-42dc-9f08-b63268c135c8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.687834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: INFO dbcounter [None req-4bf1d5de-d4b8-42dc-9f08-b63268c135c8 None None] Registered counter for database keystone Jun 16 15:58:14.707020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG oslo_db.sqlalchemy.engines [None req-4bf1d5de-d4b8-42dc-9f08-b63268c135c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 15:58:14.713244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writer thread running {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 15:58:14.834654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1d5de-d4b8-42dc-9f08-b63268c135c8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.838983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b5a80-03eb-41fa-934c-bed2fe8e9922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.839643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b5a80-03eb-41fa-934c-bed2fe8e9922 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.839935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b5a80-03eb-41fa-934c-bed2fe8e9922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.840214 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b5a80-03eb-41fa-934c-bed2fe8e9922 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.848396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b5a80-03eb-41fa-934c-bed2fe8e9922 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.901759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING keystone.common.rbac_enforcer.enforcer [None req-332b5a80-03eb-41fa-934c-bed2fe8e9922 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 15:58:14.902042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b5a80-03eb-41fa-934c-bed2fe8e9922 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.911909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 2/4] 10.222.0.88 () {58 vars in 1324 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 15:58:14.917934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:14.935235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2a5d1-8f16-45bf-8edc-72781bfc45cb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:14.939695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b684c-5613-4074-a4e3-0c1434bc41c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:14.940787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b684c-5613-4074-a4e3-0c1434bc41c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:14.941074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b684c-5613-4074-a4e3-0c1434bc41c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:14.941360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b684c-5613-4074-a4e3-0c1434bc41c5 admin admin] PATH_INFO: `/v3/projects/a913cabdc8c6449f971e776cd4ba8813` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:14.942862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b684c-5613-4074-a4e3-0c1434bc41c5 admin admin] RBAC: Authorizing `identity:get_project(project_id=a913cabdc8c6449f971e776cd4ba8813)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:14.993637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a39b684c-5613-4074-a4e3-0c1434bc41c5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 15:58:14.994002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b684c-5613-4074-a4e3-0c1434bc41c5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:14.998623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 3/5] 10.222.0.88 () {56 vars in 1207 bytes} [Thu Jun 16 15:58:14 2022] GET /identity/v3/projects/a913cabdc8c6449f971e776cd4ba8813 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:17.081186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b550e-13be-41e4-9cb8-4c0e79f0f2c7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.081763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b550e-13be-41e4-9cb8-4c0e79f0f2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.081763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b550e-13be-41e4-9cb8-4c0e79f0f2c7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.086375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 3/6] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:17.091641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb67d1-80d0-46ff-abc3-baa0bcbbc89a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.092057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb67d1-80d0-46ff-abc3-baa0bcbbc89a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.092204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb67d1-80d0-46ff-abc3-baa0bcbbc89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.111393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d3bb67d1-80d0-46ff-abc3-baa0bcbbc89a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:17.112937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d3bb67d1-80d0-46ff-abc3-baa0bcbbc89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:17.120991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 4/7] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 15:58:17.143709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b550e-13be-41e4-9cb8-4c0e79f0f2c7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.156469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b550e-13be-41e4-9cb8-4c0e79f0f2c7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.160706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd4ff7-52fd-401a-a8c7-41c128cd66a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.161346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4ff7-52fd-401a-a8c7-41c128cd66a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.161634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4ff7-52fd-401a-a8c7-41c128cd66a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.161968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4ff7-52fd-401a-a8c7-41c128cd66a8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.170434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4ff7-52fd-401a-a8c7-41c128cd66a8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:17.172216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4ff7-52fd-401a-a8c7-41c128cd66a8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:17.180976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 4/8] 10.222.0.88 () {58 vars in 1324 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 15:58:17.186347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb67d1-80d0-46ff-abc3-baa0bcbbc89a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:17.195415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb67d1-80d0-46ff-abc3-baa0bcbbc89a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:17.200197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7eb6-0011-4dbe-ab8c-277b9c36da16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:17.201067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7eb6-0011-4dbe-ab8c-277b9c36da16 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:17.201352 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7eb6-0011-4dbe-ab8c-277b9c36da16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:17.201629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7eb6-0011-4dbe-ab8c-277b9c36da16 admin admin] PATH_INFO: `/v3/users/fded449262ae4659b5d91d4b0bc2b7a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:17.203054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7eb6-0011-4dbe-ab8c-277b9c36da16 admin admin] RBAC: Authorizing `identity:get_user(user_id=fded449262ae4659b5d91d4b0bc2b7a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:17.204886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7eb6-0011-4dbe-ab8c-277b9c36da16 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:17.206520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 5/9] 10.222.0.88 () {56 vars in 1198 bytes} [Thu Jun 16 15:58:17 2022] GET /identity/v3/users/fded449262ae4659b5d91d4b0bc2b7a0 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:19.383193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eccbb2-f299-4d1d-bf16-727ac88201cd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.383747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eccbb2-f299-4d1d-bf16-727ac88201cd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.383747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eccbb2-f299-4d1d-bf16-727ac88201cd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.385129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 5/10] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:19.391286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a604-e9ff-4449-b101-bedf48e88a20 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.391784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a604-e9ff-4449-b101-bedf48e88a20 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.391784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a604-e9ff-4449-b101-bedf48e88a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.409490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9027a604-e9ff-4449-b101-bedf48e88a20 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:19.410755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9027a604-e9ff-4449-b101-bedf48e88a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:19.446120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836533a1-305c-44a6-9303-abceba723335 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.446409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836533a1-305c-44a6-9303-abceba723335 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.446688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836533a1-305c-44a6-9303-abceba723335 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.448044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 6/11] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:19.453198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.453485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.453793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.494167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG passlib.handlers.bcrypt [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80396) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 15:58:19.494383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 6/12] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:19.494665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG passlib.handlers.bcrypt [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80396) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 15:58:19.516824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a604-e9ff-4449-b101-bedf48e88a20 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.540769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.543111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:19.544740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:19.622694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a604-e9ff-4449-b101-bedf48e88a20 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.644126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 7/13] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:19.656560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.657364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.657656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.657967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.666528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.669984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:19.681590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:19.682552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:19.683383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 7/14] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:19.688518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.767765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f0b50-7bea-429f-9cb5-5800a3a004da None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.770157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.771832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.791466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455faec-6367-4688-8ad3-05590958c5de None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.806846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.807748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.808041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.808328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.831070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.831751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.832041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.832329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.833093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:19.833409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.834861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.846736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 8/15] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:19.852795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.879196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.880730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.951364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd962e35-504d-4aea-a61a-f5e3999723db None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:19.983461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 8/16] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:58:19.985313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:19.986241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:19.986531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:19.986814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] PATH_INFO: `/v3/users/fded449262ae4659b5d91d4b0bc2b7a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:19.988141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] RBAC: Authorizing `identity:get_user(user_id=fded449262ae4659b5d91d4b0bc2b7a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:19.989253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:19.989609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:19.991279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 9/17] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/users/fded449262ae4659b5d91d4b0bc2b7a0 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:19.994433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:19.999075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:20.014699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:20.091634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f163e-7f3c-465d-965c-e59d3ace24ff None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.100797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bbf74-7994-4d08-bd32-9e9ec693bf36 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.126431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.127222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.127505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.127780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.138177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.138982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.139266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.139543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.140728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.142240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.143839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 10/18] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:20.149981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:20.151283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.152760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:20.173032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:20.173930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:20.174729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:20.174729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:20.174729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:20.174729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:20.174729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 9/19] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:20.179796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:20.232091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:20.236646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:20.253321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434c6ce-9ecf-4cca-b14f-6eb3fc6f7597 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.281429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2b416-04be-4cbc-b830-bfa8adade2c7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:20.287595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0a87b-0df0-4485-bbae-968ef293f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.288262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0a87b-0df0-4485-bbae-968ef293f7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.288553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0a87b-0df0-4485-bbae-968ef293f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.288839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0a87b-0df0-4485-bbae-968ef293f7c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.290245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0a87b-0df0-4485-bbae-968ef293f7c2 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.domain.id=default, user.id=fded449262ae4659b5d91d4b0bc2b7a0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:20.290567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0a87b-0df0-4485-bbae-968ef293f7c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.292024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0a87b-0df0-4485-bbae-968ef293f7c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.298825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 11/20] 10.222.0.88 () {56 vars in 1322 bytes} [Thu Jun 16 15:58:20 2022] GET /identity/v3/role_assignments?user.id=fded449262ae4659b5d91d4b0bc2b7a0&scope.domain.id=default&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:20.316547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72e000-b5e5-46aa-acf9-23798670e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:20.317368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e000-b5e5-46aa-acf9-23798670e583 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:20.317660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e000-b5e5-46aa-acf9-23798670e583 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:20.317973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72e000-b5e5-46aa-acf9-23798670e583 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:20.318735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e000-b5e5-46aa-acf9-23798670e583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:20.319050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e000-b5e5-46aa-acf9-23798670e583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:20.320512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72e000-b5e5-46aa-acf9-23798670e583 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:20.341522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 10/21] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:22.788571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad32c4-212d-45c5-bdfe-94a5b7339722 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.789445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad32c4-212d-45c5-bdfe-94a5b7339722 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.790550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad32c4-212d-45c5-bdfe-94a5b7339722 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.792158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 12/22] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:22.797400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c310a44-b837-4014-b109-bfb842d735f5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.798212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c310a44-b837-4014-b109-bfb842d735f5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.798630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c310a44-b837-4014-b109-bfb842d735f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.827272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4c310a44-b837-4014-b109-bfb842d735f5 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:22.828771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4c310a44-b837-4014-b109-bfb842d735f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:22.896525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8810ba-5166-4f63-b713-62836d12258f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.897110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8810ba-5166-4f63-b713-62836d12258f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.897565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8810ba-5166-4f63-b713-62836d12258f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.899155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 13/23] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:22.909336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51644c8-1710-4539-9a6f-a24f765de604 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:22.909935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51644c8-1710-4539-9a6f-a24f765de604 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:22.910414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51644c8-1710-4539-9a6f-a24f765de604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:22.926248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 11/24] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:22.944465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d51644c8-1710-4539-9a6f-a24f765de604 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:22.945977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d51644c8-1710-4539-9a6f-a24f765de604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:22.951940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c310a44-b837-4014-b109-bfb842d735f5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.032535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 14/25] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:23.057669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51644c8-1710-4539-9a6f-a24f765de604 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.066083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.141640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.172701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51644c8-1710-4539-9a6f-a24f765de604 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.218445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.219533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.219821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.220100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.235367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:23.244697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c310a44-b837-4014-b109-bfb842d735f5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:23.250344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:23.251266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:23.252490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 15/26] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:23.261752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.354761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.356363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.357041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.385357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.386391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb52eb8-0505-4531-90ea-ef034beda051 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.388368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.426088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a338d-a700-4d14-a71c-41443bd3739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.426843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a338d-a700-4d14-a71c-41443bd3739d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.427134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a338d-a700-4d14-a71c-41443bd3739d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.427417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a338d-a700-4d14-a71c-41443bd3739d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.428179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a338d-a700-4d14-a71c-41443bd3739d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:23.428491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a338d-a700-4d14-a71c-41443bd3739d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.429932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a338d-a700-4d14-a71c-41443bd3739d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.431967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:58:23.432675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:23.433610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:23.438782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:58:23.438782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:58:23.438782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:58:23.438782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:23.440693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 12/27] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:23.440958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 16/28] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:23.446191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c78181-ca51-4104-96f0-f34c8a69b3ac None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.446336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.446602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c78181-ca51-4104-96f0-f34c8a69b3ac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.446818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c78181-ca51-4104-96f0-f34c8a69b3ac None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.448169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 17/29] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:23.449675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c78181-ca51-4104-96f0-f34c8a69b3ac None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.568061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c78181-ca51-4104-96f0-f34c8a69b3ac None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.607715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c947e0bb-2e31-456c-bc12-0bfdc34cfc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.608457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947e0bb-2e31-456c-bc12-0bfdc34cfc5b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.608757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947e0bb-2e31-456c-bc12-0bfdc34cfc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.609039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c947e0bb-2e31-456c-bc12-0bfdc34cfc5b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.613751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947e0bb-2e31-456c-bc12-0bfdc34cfc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:23.613751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947e0bb-2e31-456c-bc12-0bfdc34cfc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.613751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c947e0bb-2e31-456c-bc12-0bfdc34cfc5b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.633907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 18/30] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:23.636083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2150e2-2c03-4966-b0f8-1f4a440da777 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.636382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2150e2-2c03-4966-b0f8-1f4a440da777 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.636665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2150e2-2c03-4966-b0f8-1f4a440da777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.662724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4c2150e2-2c03-4966-b0f8-1f4a440da777 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:23.664498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c2150e2-2c03-4966-b0f8-1f4a440da777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:23.715307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.723932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e04ba-a0d8-4ba0-83ba-0615de900c88 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.730123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:23.749094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 19/31] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:23.749952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2150e2-2c03-4966-b0f8-1f4a440da777 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.761936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e7713-a4c2-405b-a620-43230603bce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.762883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7713-a4c2-405b-a620-43230603bce2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.763173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7713-a4c2-405b-a620-43230603bce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.763454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7713-a4c2-405b-a620-43230603bce2 None admin] PATH_INFO: `/v3/users/fded449262ae4659b5d91d4b0bc2b7a0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.765124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e7713-a4c2-405b-a620-43230603bce2 None admin] RBAC: Authorizing `identity:get_user(user_id=fded449262ae4659b5d91d4b0bc2b7a0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.766741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e7713-a4c2-405b-a620-43230603bce2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.768566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 13/32] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/users/fded449262ae4659b5d91d4b0bc2b7a0 => generated 247 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:23.769942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257d6df-f706-416b-83f6-03796ac08517 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.770226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257d6df-f706-416b-83f6-03796ac08517 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.774791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257d6df-f706-416b-83f6-03796ac08517 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.776517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 14/33] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:23.777250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257d6df-f706-416b-83f6-03796ac08517 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:23.859893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2150e2-2c03-4966-b0f8-1f4a440da777 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.892055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257d6df-f706-416b-83f6-03796ac08517 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:23.895958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.896628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.896913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.897190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.897953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.899574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.930866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546b2cd-6a1c-47c1-8cdc-4a453ca9a346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:23.932259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546b2cd-6a1c-47c1-8cdc-4a453ca9a346 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.932549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546b2cd-6a1c-47c1-8cdc-4a453ca9a346 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.932879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546b2cd-6a1c-47c1-8cdc-4a453ca9a346 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.934122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546b2cd-6a1c-47c1-8cdc-4a453ca9a346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:23.935666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546b2cd-6a1c-47c1-8cdc-4a453ca9a346 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:23.937415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 15/34] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:23.943242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a13674-5355-4c15-9e22-5309d7607691 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:23.943532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a13674-5355-4c15-9e22-5309d7607691 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:23.943820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a13674-5355-4c15-9e22-5309d7607691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:23.961863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c2a13674-5355-4c15-9e22-5309d7607691 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:23.963677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2a13674-5355-4c15-9e22-5309d7607691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:23.995002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.043888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 16/35] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:24.044951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a13674-5355-4c15-9e22-5309d7607691 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.153841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a13674-5355-4c15-9e22-5309d7607691 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.183102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 20/36] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 15:58:23 2022] POST /identity/v3/projects => generated 301 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:24.190126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.198012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.198992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.199450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.199866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.200820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.204883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.280243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.399248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 17/37] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 15:58:24 2022] POST /identity/v3/roles => generated 219 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:24.402577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.419863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b666dc-1858-4402-a41a-d6f709f923ad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.432836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.462185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9d3f0-669f-41f6-9843-4844b02dc866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.462780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9d3f0-669f-41f6-9843-4844b02dc866 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.463076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9d3f0-669f-41f6-9843-4844b02dc866 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.463366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9d3f0-669f-41f6-9843-4844b02dc866 None admin] PATH_INFO: `/v3/domains/default/users/fded449262ae4659b5d91d4b0bc2b7a0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.467020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9d3f0-669f-41f6-9843-4844b02dc866 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fded449262ae4659b5d91d4b0bc2b7a0, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.468598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9d3f0-669f-41f6-9843-4844b02dc866 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.497737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 21/38] 10.222.0.88 () {58 vars in 1361 bytes} [Thu Jun 16 15:58:24 2022] PUT /identity/v3/domains/default/users/fded449262ae4659b5d91d4b0bc2b7a0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:24.545636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576e41-1156-43e4-9ede-5b591973b049 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:24.592036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de6249-eab2-4a94-ae62-0feb17abee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:24.592745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de6249-eab2-4a94-ae62-0feb17abee80 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.593037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de6249-eab2-4a94-ae62-0feb17abee80 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.593324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de6249-eab2-4a94-ae62-0feb17abee80 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.594157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de6249-eab2-4a94-ae62-0feb17abee80 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:24.595928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de6249-eab2-4a94-ae62-0feb17abee80 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:24.613761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 18/39] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 15:58:24 2022] POST /identity/v3/roles => generated 225 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:24.639682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10990-5b88-4c87-b78d-44f9532a6ee0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.640023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10990-5b88-4c87-b78d-44f9532a6ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.640261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10990-5b88-4c87-b78d-44f9532a6ee0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.641968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 22/40] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:24.660143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47030ee-79ae-4059-a85f-da9cbf35e127 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.660436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47030ee-79ae-4059-a85f-da9cbf35e127 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.660735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47030ee-79ae-4059-a85f-da9cbf35e127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.682944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c47030ee-79ae-4059-a85f-da9cbf35e127 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:24.684726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c47030ee-79ae-4059-a85f-da9cbf35e127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:24.834541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177aa2-d8a4-4889-a53e-a7b60591e05f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.834841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177aa2-d8a4-4889-a53e-a7b60591e05f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.835130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53177aa2-d8a4-4889-a53e-a7b60591e05f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.836546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 23/41] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:24.842716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef84b2-d220-43f3-9e58-7566f5dfadda None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:24.843017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef84b2-d220-43f3-9e58-7566f5dfadda None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:24.843306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef84b2-d220-43f3-9e58-7566f5dfadda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:24.865766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fdef84b2-d220-43f3-9e58-7566f5dfadda None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:24.867438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fdef84b2-d220-43f3-9e58-7566f5dfadda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:24.888081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 19/42] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:24.913319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47030ee-79ae-4059-a85f-da9cbf35e127 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:24.930939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:24.965559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 24/43] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:24.988217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef84b2-d220-43f3-9e58-7566f5dfadda None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:25.017491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:25.022162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47030ee-79ae-4059-a85f-da9cbf35e127 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.071336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.072544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.072852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.073139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.073947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.075349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.091502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 20/44] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 15:58:24 2022] POST /identity/v3/roles => generated 223 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:58:25.112663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef84b2-d220-43f3-9e58-7566f5dfadda None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.159981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe1d66-8cef-4c9a-8f6e-913eb9413416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.160809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1d66-8cef-4c9a-8f6e-913eb9413416 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.161148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1d66-8cef-4c9a-8f6e-913eb9413416 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.161481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe1d66-8cef-4c9a-8f6e-913eb9413416 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.162811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe1d66-8cef-4c9a-8f6e-913eb9413416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.164495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe1d66-8cef-4c9a-8f6e-913eb9413416 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.167000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 25/45] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:25.172550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:25.260288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:25.293187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb2d24-ef22-4220-8d88-508573462406 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:25.330935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0414c2b-b024-4ca6-9784-91ae392b4ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:25.331617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0414c2b-b024-4ca6-9784-91ae392b4ca1 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:25.331940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0414c2b-b024-4ca6-9784-91ae392b4ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:25.332223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0414c2b-b024-4ca6-9784-91ae392b4ca1 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:25.333033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0414c2b-b024-4ca6-9784-91ae392b4ca1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:25.334524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0414c2b-b024-4ca6-9784-91ae392b4ca1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:25.362985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 21/46] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 15:58:25 2022] POST /identity/v3/projects => generated 312 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:26.838815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdf7f-cd1c-4d80-acb7-23f9190f953d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:26.839334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdf7f-cd1c-4d80-acb7-23f9190f953d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:26.839418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdf7f-cd1c-4d80-acb7-23f9190f953d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:26.840758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 26/47] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:26.846059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b57c3-5842-4a45-a751-54e90fc765e3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:26.846449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b57c3-5842-4a45-a751-54e90fc765e3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:26.846639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b57c3-5842-4a45-a751-54e90fc765e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:26.864634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cf4b57c3-5842-4a45-a751-54e90fc765e3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:26.866607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cf4b57c3-5842-4a45-a751-54e90fc765e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:26.934540 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 22/48] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:26.957011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fdf7f-cd1c-4d80-acb7-23f9190f953d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:27.054380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:27.061906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fdf7f-cd1c-4d80-acb7-23f9190f953d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:27.095675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:27.096391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.096675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.096952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.107409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:27.114865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:27.115750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:27.116516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 27/49] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:27.121914 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b57c3-5842-4a45-a751-54e90fc765e3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:27.177463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:27.222282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b57c3-5842-4a45-a751-54e90fc765e3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:27.255470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:27.256180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.256475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.256756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.257513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:27.257857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:27.259394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:27.272573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 23/50] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:27.278958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:27.379331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012c19-d04a-4d91-b82d-8fd5d2052071 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:27.413461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cbde0-b904-4f75-9fbe-b9ce2c176c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:27.414392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cbde0-b904-4f75-9fbe-b9ce2c176c71 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.414683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cbde0-b904-4f75-9fbe-b9ce2c176c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.414963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cbde0-b904-4f75-9fbe-b9ce2c176c71 None admin] PATH_INFO: `/v3/users/fded449262ae4659b5d91d4b0bc2b7a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.416293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cbde0-b904-4f75-9fbe-b9ce2c176c71 None admin] RBAC: Authorizing `identity:get_user(user_id=fded449262ae4659b5d91d4b0bc2b7a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:27.417869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329cbde0-b904-4f75-9fbe-b9ce2c176c71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:27.419618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 28/51] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:27 2022] GET /identity/v3/users/fded449262ae4659b5d91d4b0bc2b7a0 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:27.425378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:27.486973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ec47-f974-4d9b-a53f-194988d93330 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.487262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ec47-f974-4d9b-a53f-194988d93330 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.487552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781ec47-f974-4d9b-a53f-194988d93330 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.489371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 29/52] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:27.494316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a12f6d-bbe5-4003-8c33-9729ae76b8f9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.494584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a12f6d-bbe5-4003-8c33-9729ae76b8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.494863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a12f6d-bbe5-4003-8c33-9729ae76b8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.511421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-54a12f6d-bbe5-4003-8c33-9729ae76b8f9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:27.512608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54a12f6d-bbe5-4003-8c33-9729ae76b8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:27.530652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:27.537215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f9959-6ef8-4267-ac19-53d4bd54622b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:27.554500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:27.572038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:27.573449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.573761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.574040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.575241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:27.576720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:27.578629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 24/53] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:27.581029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 30/54] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:27.584946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:27.604401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a12f6d-bbe5-4003-8c33-9729ae76b8f9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:27.685122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3d6b8-2691-4516-b204-0a3b02d7844a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:27.703298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a12f6d-bbe5-4003-8c33-9729ae76b8f9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:27.720488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:27.721301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.721598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.721907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.722748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.domain.id=default, user.id=fded449262ae4659b5d91d4b0bc2b7a0) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:27.723067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:27.724586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:27.733320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 25/55] 10.222.0.88 () {56 vars in 1322 bytes} [Thu Jun 16 15:58:27 2022] GET /identity/v3/role_assignments?user.id=fded449262ae4659b5d91d4b0bc2b7a0&scope.domain.id=default&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:27.738718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545d622-886f-4ca5-b4f0-f0795e4e2373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:27.739544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545d622-886f-4ca5-b4f0-f0795e4e2373 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.739869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545d622-886f-4ca5-b4f0-f0795e4e2373 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.740264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545d622-886f-4ca5-b4f0-f0795e4e2373 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.741585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545d622-886f-4ca5-b4f0-f0795e4e2373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:27.743145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545d622-886f-4ca5-b4f0-f0795e4e2373 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:27.745423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 31/56] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:27.751507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:27.806105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:27.860215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793428a-daf1-4dee-b812-87c632d3345a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:27.897827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f4d85-9a0e-4a80-9631-15f5484559e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:27.898510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4d85-9a0e-4a80-9631-15f5484559e8 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:27.898805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4d85-9a0e-4a80-9631-15f5484559e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:27.899712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f4d85-9a0e-4a80-9631-15f5484559e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:27.899814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f4d85-9a0e-4a80-9631-15f5484559e8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:27.901835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f4d85-9a0e-4a80-9631-15f5484559e8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:27.928243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 26/57] 10.222.0.88 () {60 vars in 1140 bytes} [Thu Jun 16 15:58:27 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:30.043268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b86c-aad5-4ab2-8101-8506a663c3fb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.043782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b86c-aad5-4ab2-8101-8506a663c3fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.043894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b86c-aad5-4ab2-8101-8506a663c3fb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.045202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 32/58] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:30.050530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb2e7d-c8ce-4d21-b51a-433c68e6f317 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.050917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb2e7d-c8ce-4d21-b51a-433c68e6f317 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.051106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb2e7d-c8ce-4d21-b51a-433c68e6f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.070140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-eedb2e7d-c8ce-4d21-b51a-433c68e6f317 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:30.071345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eedb2e7d-c8ce-4d21-b51a-433c68e6f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:30.137381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 27/59] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:30.161368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b86c-aad5-4ab2-8101-8506a663c3fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:30.186758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:30.261701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b86c-aad5-4ab2-8101-8506a663c3fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:30.294594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3a0d7c-f882-4c14-a3a3-caf6a03b2891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:30.295307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0d7c-f882-4c14-a3a3-caf6a03b2891 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.295595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0d7c-f882-4c14-a3a3-caf6a03b2891 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.295877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a0d7c-f882-4c14-a3a3-caf6a03b2891 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.297049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a0d7c-f882-4c14-a3a3-caf6a03b2891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:30.298521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3a0d7c-f882-4c14-a3a3-caf6a03b2891 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:30.300135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 33/60] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:30.305797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb2e7d-c8ce-4d21-b51a-433c68e6f317 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:30.336066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:30.403859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb2e7d-c8ce-4d21-b51a-433c68e6f317 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:30.436523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a5d5a-f3cb-4e5a-b0e9-ffcd2ff40e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:30.437195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5d5a-f3cb-4e5a-b0e9-ffcd2ff40e94 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:30.437487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5d5a-f3cb-4e5a-b0e9-ffcd2ff40e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:30.437799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a5d5a-f3cb-4e5a-b0e9-ffcd2ff40e94 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:30.438508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a5d5a-f3cb-4e5a-b0e9-ffcd2ff40e94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:30.439980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a5d5a-f3cb-4e5a-b0e9-ffcd2ff40e94 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:30.465425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 28/61] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:30 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:33.798745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501188e-56e8-4d6f-b024-0df5ebda8be1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.799486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501188e-56e8-4d6f-b024-0df5ebda8be1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.799486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501188e-56e8-4d6f-b024-0df5ebda8be1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.801039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 34/62] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:33.806411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cc49-513a-4db5-b17f-715057d888c8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.806878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cc49-513a-4db5-b17f-715057d888c8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.806978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12cc49-513a-4db5-b17f-715057d888c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.830609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1b12cc49-513a-4db5-b17f-715057d888c8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:33.831848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b12cc49-513a-4db5-b17f-715057d888c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:33.902913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232fc0c-ae71-409e-bbca-bde4c258ebb0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.903545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232fc0c-ae71-409e-bbca-bde4c258ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.904052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232fc0c-ae71-409e-bbca-bde4c258ebb0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.905653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 35/63] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:33.911020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb3900-fc2c-42cb-87e3-f3e972862176 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.911570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb3900-fc2c-42cb-87e3-f3e972862176 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.912054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb3900-fc2c-42cb-87e3-f3e972862176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.943387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-31cb3900-fc2c-42cb-87e3-f3e972862176 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:33.944567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-31cb3900-fc2c-42cb-87e3-f3e972862176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:33.944895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 29/64] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:33.946394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67744ae0-4edd-4d16-b318-0fb8682bfdcb None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.946688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67744ae0-4edd-4d16-b318-0fb8682bfdcb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.946976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67744ae0-4edd-4d16-b318-0fb8682bfdcb None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.948284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 30/65] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:33.954015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f63ec-acac-4532-a6be-4ce343971f20 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:33.954732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f63ec-acac-4532-a6be-4ce343971f20 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:33.955161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f63ec-acac-4532-a6be-4ce343971f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:33.972505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-879f63ec-acac-4532-a6be-4ce343971f20 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:33.973713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-879f63ec-acac-4532-a6be-4ce343971f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:33.988699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.047139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 36/66] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:34.049018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb3900-fc2c-42cb-87e3-f3e972862176 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.052243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 31/67] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:34.070876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f63ec-acac-4532-a6be-4ce343971f20 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.090668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.177258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb3900-fc2c-42cb-87e3-f3e972862176 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.190260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f63ec-acac-4532-a6be-4ce343971f20 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.215061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.216251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.216742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.217224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.230097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.230903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:34.231880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.232309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.232723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:34.239771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.240713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:34.241666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:34.243922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 37/68] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:34.244665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.251060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:34.260708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:34.263028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 32/69] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:34.263653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.278153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.347573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.368347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b233b9f-ee44-45a5-903f-5ded0d81c988 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.383518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f20799-d605-4169-a400-026150cad3e8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.412350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11d46c-b377-4871-a8cc-baa734adff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.413430 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11d46c-b377-4871-a8cc-baa734adff50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.413919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11d46c-b377-4871-a8cc-baa734adff50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.414224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11d46c-b377-4871-a8cc-baa734adff50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.420623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.421306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.421598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.421916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.422685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.423006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.424428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.426516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11d46c-b377-4871-a8cc-baa734adff50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:34.433607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 33/70] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.434335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-dd11d46c-b377-4871-a8cc-baa734adff50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:34.436255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:34.437086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:34.437890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:34.444063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 38/71] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:34.444063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edf715-6f7c-444d-95ad-99310a8cba76 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.444063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edf715-6f7c-444d-95ad-99310a8cba76 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.444063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edf715-6f7c-444d-95ad-99310a8cba76 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.444063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 39/72] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:34.444063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edf715-6f7c-444d-95ad-99310a8cba76 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.529486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.562390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44edf715-6f7c-444d-95ad-99310a8cba76 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.564686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bec16-dda8-4ce8-97b8-a3811c7bf7e0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.596773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.597748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.597972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.598252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] PATH_INFO: `/v3/users/fded449262ae4659b5d91d4b0bc2b7a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.599581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] RBAC: Authorizing `identity:get_user(user_id=fded449262ae4659b5d91d4b0bc2b7a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.599686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44571c4-6bdc-4c5e-91c8-510d9f997040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.600254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44571c4-6bdc-4c5e-91c8-510d9f997040 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.600548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44571c4-6bdc-4c5e-91c8-510d9f997040 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.600915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44571c4-6bdc-4c5e-91c8-510d9f997040 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.601524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.601662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44571c4-6bdc-4c5e-91c8-510d9f997040 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.602085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44571c4-6bdc-4c5e-91c8-510d9f997040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.603385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 40/73] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/users/fded449262ae4659b5d91d4b0bc2b7a0 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.603470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44571c4-6bdc-4c5e-91c8-510d9f997040 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.607284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.619751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 34/74] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.619751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9f2a2-3f9d-458f-93a4-0bdb6ae71502 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.619751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9f2a2-3f9d-458f-93a4-0bdb6ae71502 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.619751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9f2a2-3f9d-458f-93a4-0bdb6ae71502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.620094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.636606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5ab9f2a2-3f9d-458f-93a4-0bdb6ae71502 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:34.637805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5ab9f2a2-3f9d-458f-93a4-0bdb6ae71502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:34.706033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 35/75] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:34.707375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2a3e1-ea92-4ab5-a64c-486ae737e084 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.707667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2a3e1-ea92-4ab5-a64c-486ae737e084 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.707949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2a3e1-ea92-4ab5-a64c-486ae737e084 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.709237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 36/76] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:34.713585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2a3e1-ea92-4ab5-a64c-486ae737e084 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.729106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435e3e5-a7ed-404d-9875-16fdcffdbf82 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.765778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.766455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.766752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.767044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.767797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:34.768111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.769531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.777763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 41/77] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.778565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:34.788400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.828934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2a3e1-ea92-4ab5-a64c-486ae737e084 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.851061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:34.872604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.873268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.873564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.873868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.882728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6856471-a52a-4323-9588-91cccc7da57b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:34.919718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d17c0-01c2-4e9f-b863-c748fd4806dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:34.920602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d17c0-01c2-4e9f-b863-c748fd4806dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.920895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d17c0-01c2-4e9f-b863-c748fd4806dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.921176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d17c0-01c2-4e9f-b863-c748fd4806dc None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.935920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d17c0-01c2-4e9f-b863-c748fd4806dc None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.937515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d17c0-01c2-4e9f-b863-c748fd4806dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.939261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 42/78] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:34.940617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d778130-ff02-4ece-b99b-07c066d2c152 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:34.941042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d778130-ff02-4ece-b99b-07c066d2c152 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:34.941297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d778130-ff02-4ece-b99b-07c066d2c152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:34.944848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:34.946398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:34.960308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2d778130-ff02-4ece-b99b-07c066d2c152 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:34.961501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d778130-ff02-4ece-b99b-07c066d2c152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:35.006712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.043080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 43/79] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:35.044518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d778130-ff02-4ece-b99b-07c066d2c152 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.047576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 37/80] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:35.048407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.119114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.157691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf78abc-b296-42b8-b319-29cb306412ad None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.163400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d778130-ff02-4ece-b99b-07c066d2c152 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.192319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.193268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.193555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.193863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.195363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.196559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.198361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 38/81] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.198536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.199130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.199356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.200207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.200388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.212861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:35.223406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:35.224379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:35.225134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 44/82] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:35.226360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.240280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.315281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d55821-08d0-4475-940f-9a6ec20c2636 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.332486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efc83c-41c2-4ffc-b379-2f72082bfb9a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.350786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.351519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.351813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.352057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.352182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.374120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.375214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.375507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.375793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.377124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.378635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.387850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 45/83] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.387850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.423736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.425240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.454474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.488063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dd92b-d0fb-4e32-9b1d-6252688d1ba0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.525112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.525936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.526227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.526502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.528194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.529672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.531426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 46/84] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.532195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.532730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 39/85] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:35.533848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.590325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.640735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec398740-b62a-4206-932d-a196181e6e9f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.642892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de495b2-9a65-4bb3-bfd5-d09234f99afe None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.675011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.679170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.679830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.680120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.680401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.682623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.683328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.683622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.684489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.685329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:35.685642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.687070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.695846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 40/86] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.696690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.752404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.753962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.798536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245df9a-2e28-4dcc-9d71-ccccc17bbaf9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.807417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.833204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:35.833901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:35.834187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:35.834466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:35.835295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=fded449262ae4659b5d91d4b0bc2b7a0) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:35.835611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:35.837055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:35.843005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 41/87] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/role_assignments?user.id=fded449262ae4659b5d91d4b0bc2b7a0&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:35.843755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.863289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 47/88] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:35.863289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:35.915228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:35.970074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbe2cd-f7d1-4676-b275-26a489fc15ce None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:35.981787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2645d3-08dd-428b-94cc-4ae41fbc04fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.007162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.007981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.008284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.008662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.009401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.012404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.023310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea08be5-a048-4653-a400-03db10be675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.024094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea08be5-a048-4653-a400-03db10be675b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.024403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea08be5-a048-4653-a400-03db10be675b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.024775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea08be5-a048-4653-a400-03db10be675b None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.026105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea08be5-a048-4653-a400-03db10be675b None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.028193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea08be5-a048-4653-a400-03db10be675b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.029975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 48/89] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:35 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.042553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 42/90] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:35 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:36.042553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.042804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea08be5-a048-4653-a400-03db10be675b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.071946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.173492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea08be5-a048-4653-a400-03db10be675b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.195685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d30ba0-bf0c-4059-8799-998f0357f68c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.215022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.217637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.218758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.219284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.219935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.222256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.223950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.226180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 49/91] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.227253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.252141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.253029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.253318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.253602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.255029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.256596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.258419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 43/92] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.259162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.338126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.352414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1825b0-4064-4d78-be6c-03a8e6408212 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.386907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ad927-c1e3-4b94-9502-80a401fed54b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.398548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.398889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.399484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.399796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.400646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=9936dac6582a4f55a12ec17571df8f3e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:36.400971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.402879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.410622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 50/93] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/role_assignments?user.id=9936dac6582a4f55a12ec17571df8f3e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.412643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.440932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.441544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.441890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.442175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.508855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.533867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.535383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.556213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5d079-ec09-474e-bc9f-11ef78d0c845 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.600982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.602139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.602681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.603532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.604693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] RBAC: Adding query filter params (role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=9936dac6582a4f55a12ec17571df8f3e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:36.605264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.607097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.614241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 51/94] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/role_assignments?user.id=9936dac6582a4f55a12ec17571df8f3e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.650017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 44/95] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:36.657003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.682625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.828923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1a8d7-459d-4b96-b506-ba1b1aad6b37 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:36.866026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f5519-3a43-4564-80bb-c88dbb808447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:36.867148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5519-3a43-4564-80bb-c88dbb808447 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:36.867476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5519-3a43-4564-80bb-c88dbb808447 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:36.867762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5519-3a43-4564-80bb-c88dbb808447 None admin] PATH_INFO: `/v3/projects/9172b93641c341519b19ef9fda02965e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:36.869156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f5519-3a43-4564-80bb-c88dbb808447 None admin] RBAC: Authorizing `identity:get_project(project_id=9172b93641c341519b19ef9fda02965e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:36.871116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f5519-3a43-4564-80bb-c88dbb808447 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:36.873434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 52/96] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/projects/9172b93641c341519b19ef9fda02965e => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:36.879675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:36.970555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:36.990982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9603b5-e961-4702-8d1c-c3f151ec5c0a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:37.028223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0ff70-0089-4f94-8f0f-356c1232c682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:37.028902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0ff70-0089-4f94-8f0f-356c1232c682 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:37.029196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0ff70-0089-4f94-8f0f-356c1232c682 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:37.029484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0ff70-0089-4f94-8f0f-356c1232c682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:37.030380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0ff70-0089-4f94-8f0f-356c1232c682 None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=9172b93641c341519b19ef9fda02965e, user.id=9936dac6582a4f55a12ec17571df8f3e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:37.030705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0ff70-0089-4f94-8f0f-356c1232c682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:37.032252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0ff70-0089-4f94-8f0f-356c1232c682 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:37.038911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 45/97] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:36 2022] GET /identity/v3/role_assignments?user.id=9936dac6582a4f55a12ec17571df8f3e&scope.project.id=9172b93641c341519b19ef9fda02965e&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:38.427102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406220-f95c-4e14-818b-1653a96f016e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.427855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406220-f95c-4e14-818b-1653a96f016e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.427855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406220-f95c-4e14-818b-1653a96f016e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.430933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 53/98] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:38.440290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cb2e5-bd96-4b0d-aaf0-d09b6b8d07e7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.441074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cb2e5-bd96-4b0d-aaf0-d09b6b8d07e7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.441367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cb2e5-bd96-4b0d-aaf0-d09b6b8d07e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.470225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0b4cb2e5-bd96-4b0d-aaf0-d09b6b8d07e7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:38.472460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0b4cb2e5-bd96-4b0d-aaf0-d09b6b8d07e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:38.591488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 46/99] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:38.642256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89406220-f95c-4e14-818b-1653a96f016e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.752503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:38.774689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89406220-f95c-4e14-818b-1653a96f016e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:38.820831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:38.821654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:38.821969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:38.822253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:38.842614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:38.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:38.851147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:38.851906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:38.852504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 54/100] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:38.858000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cb2e5-bd96-4b0d-aaf0-d09b6b8d07e7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:38.952779 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:38.984987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cb2e5-bd96-4b0d-aaf0-d09b6b8d07e7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.035639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.036249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.036542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.036828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.037600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:39.037943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.039428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.053652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 47/101] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:39.059149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.202849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf7d74-f35f-43a3-95db-f39d53bc2462 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.251080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.251973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.252265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.252548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] PATH_INFO: `/v3/users/fded449262ae4659b5d91d4b0bc2b7a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.254508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] RBAC: Authorizing `identity:get_user(user_id=fded449262ae4659b5d91d4b0bc2b7a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.255993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.257676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 55/102] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/users/fded449262ae4659b5d91d4b0bc2b7a0 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:39.263956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.395749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a2ecc-034c-4494-b4df-611b9b5764d0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.435491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b702387-9d7b-45f5-b5f8-21b88891ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.436441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b702387-9d7b-45f5-b5f8-21b88891ef0b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.436729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b702387-9d7b-45f5-b5f8-21b88891ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.437010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b702387-9d7b-45f5-b5f8-21b88891ef0b None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.438282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b702387-9d7b-45f5-b5f8-21b88891ef0b None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.439728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b702387-9d7b-45f5-b5f8-21b88891ef0b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.441742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 48/103] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:39.451317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.489963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.603614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086b3ae-e6e8-4c1f-b326-7f1c8d2333ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:39.657672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7ecfc-61fd-4226-a551-bc831d379f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:39.658544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ecfc-61fd-4226-a551-bc831d379f94 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.658907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ecfc-61fd-4226-a551-bc831d379f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.659287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7ecfc-61fd-4226-a551-bc831d379f94 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a/users/fded449262ae4659b5d91d4b0bc2b7a0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.661350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7ecfc-61fd-4226-a551-bc831d379f94 None admin] RBAC: Authorizing `identity:create_grant(project_id=caaea6a33f9840079019e777b132555a, user_id=fded449262ae4659b5d91d4b0bc2b7a0, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:39.662862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7ecfc-61fd-4226-a551-bc831d379f94 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:39.679434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 56/104] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:39 2022] PUT /identity/v3/projects/caaea6a33f9840079019e777b132555a/users/fded449262ae4659b5d91d4b0bc2b7a0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:39.719041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91942a-a93c-4eb0-888e-653def5a2778 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.719226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91942a-a93c-4eb0-888e-653def5a2778 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.719468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91942a-a93c-4eb0-888e-653def5a2778 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.721130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 49/105] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:39.726233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d59a1-9b7c-4c68-ad5f-e3dd73395774 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.726538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d59a1-9b7c-4c68-ad5f-e3dd73395774 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.726829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d59a1-9b7c-4c68-ad5f-e3dd73395774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.751657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3f1d59a1-9b7c-4c68-ad5f-e3dd73395774 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:39.752937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f1d59a1-9b7c-4c68-ad5f-e3dd73395774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:39.858018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 57/106] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:39.891190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91942a-a93c-4eb0-888e-653def5a2778 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:39.897065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:39.979260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f10f-81b5-4d3c-8020-a1203419999b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.979576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f10f-81b5-4d3c-8020-a1203419999b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.979764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015f10f-81b5-4d3c-8020-a1203419999b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:39.982028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 58/107] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:39.987813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43206da-1f44-4df2-afbf-c2cd90cb1c0e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:39.988117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43206da-1f44-4df2-afbf-c2cd90cb1c0e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:39.988430 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43206da-1f44-4df2-afbf-c2cd90cb1c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.001985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.026654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a43206da-1f44-4df2-afbf-c2cd90cb1c0e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:40.027889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a43206da-1f44-4df2-afbf-c2cd90cb1c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:40.091512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91942a-a93c-4eb0-888e-653def5a2778 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.146234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 59/108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:40.146234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78746098-b28e-41dd-b8e7-88d2c8e7bdc6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.146234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78746098-b28e-41dd-b8e7-88d2c8e7bdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.146234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78746098-b28e-41dd-b8e7-88d2c8e7bdc6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.146234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 60/109] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:40.146643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.146643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.146643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.146870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.154601 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dca2b-62e2-4bed-a10e-789488f8b5c8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.155097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dca2b-62e2-4bed-a10e-789488f8b5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.155532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dca2b-62e2-4bed-a10e-789488f8b5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.170360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:40.180198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.181136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:40.181955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:40.185543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-412dca2b-62e2-4bed-a10e-789488f8b5c8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:40.186322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 50/110] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.187177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.187634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-412dca2b-62e2-4bed-a10e-789488f8b5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:40.264853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 61/111] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:40.265854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dca2b-62e2-4bed-a10e-789488f8b5c8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.321491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.370341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.378380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412dca2b-62e2-4bed-a10e-789488f8b5c8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.412718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abf7be-af0e-4f40-9f69-70c1f5573148 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.438691 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.439652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.440193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.441420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.442361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:40.442982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.446058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.465156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 62/112] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:40.466419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.479393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a50f85-0f24-4769-a23f-005904e363f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.480199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a50f85-0f24-4769-a23f-005904e363f2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.480488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a50f85-0f24-4769-a23f-005904e363f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.480771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a50f85-0f24-4769-a23f-005904e363f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.491500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a50f85-0f24-4769-a23f-005904e363f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-53a50f85-0f24-4769-a23f-005904e363f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:40.500213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:40.501122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:40.502310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 51/113] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:40.503016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f7150-c250-461e-8e63-fda4704f15b2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.503016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f7150-c250-461e-8e63-fda4704f15b2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.503308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f7150-c250-461e-8e63-fda4704f15b2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.504817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 52/114] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:40.514171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f7150-c250-461e-8e63-fda4704f15b2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.594819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4563b-6d0d-4934-b2f1-9f9f9f9a1fb1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.620903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f7150-c250-461e-8e63-fda4704f15b2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.629455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.634032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.635220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.635683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.636134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.637756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.639443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.641457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 63/115] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:40.642458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.652118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.664155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4787336f-bc1e-46d7-976c-784f9f97d10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.665069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787336f-bc1e-46d7-976c-784f9f97d10f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.665372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787336f-bc1e-46d7-976c-784f9f97d10f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.665645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4787336f-bc1e-46d7-976c-784f9f97d10f None admin] PATH_INFO: `/v3/users/2be244ee2ef6473ba7ca7d3ca7edc3af` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.683683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787336f-bc1e-46d7-976c-784f9f97d10f None admin] RBAC: Authorizing `identity:get_user(user_id=2be244ee2ef6473ba7ca7d3ca7edc3af)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.684956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4787336f-bc1e-46d7-976c-784f9f97d10f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.686851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 53/116] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/users/2be244ee2ef6473ba7ca7d3ca7edc3af => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:40.688141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604932d-9d02-4ca5-a25b-e809f80c82b0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.688433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604932d-9d02-4ca5-a25b-e809f80c82b0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.688717 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604932d-9d02-4ca5-a25b-e809f80c82b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.711130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d604932d-9d02-4ca5-a25b-e809f80c82b0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:40.712331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d604932d-9d02-4ca5-a25b-e809f80c82b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:40.764486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86899193-dde1-470b-abf6-4956c02a4e1e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.781471 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 54/117] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:40.786614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604932d-9d02-4ca5-a25b-e809f80c82b0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.803796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.804505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.804800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.805149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.805976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:40.806473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.807930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.817772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 64/118] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:40.822579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.887664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.906150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604932d-9d02-4ca5-a25b-e809f80c82b0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.920785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:40.940563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b92c-bf89-49fe-94e5-3455cb18e689 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:40.944067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743b9a1-5677-471b-ae7f-a0319f56963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:40.944744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743b9a1-5677-471b-ae7f-a0319f56963d None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.945033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743b9a1-5677-471b-ae7f-a0319f56963d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.945317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743b9a1-5677-471b-ae7f-a0319f56963d None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.946122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743b9a1-5677-471b-ae7f-a0319f56963d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:40.947560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743b9a1-5677-471b-ae7f-a0319f56963d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:40.975536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 55/119] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 15:58:40 2022] POST /identity/v3/projects => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:40.977194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aabc6-1e0e-4d1b-88c5-84566fa899dc None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:40.977489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aabc6-1e0e-4d1b-88c5-84566fa899dc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:40.978139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aabc6-1e0e-4d1b-88c5-84566fa899dc None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:40.979662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 56/120] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:40.984570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90aabc6-1e0e-4d1b-88c5-84566fa899dc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:40.999216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56652d7-dbb7-461f-a3c9-5ef03fc7ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.000932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56652d7-dbb7-461f-a3c9-5ef03fc7ab0d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.001537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56652d7-dbb7-461f-a3c9-5ef03fc7ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.002094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56652d7-dbb7-461f-a3c9-5ef03fc7ab0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.108883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90aabc6-1e0e-4d1b-88c5-84566fa899dc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.115069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56652d7-dbb7-461f-a3c9-5ef03fc7ab0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.116883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56652d7-dbb7-461f-a3c9-5ef03fc7ab0d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.172388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8788110-186c-492f-a250-e414eb15184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.172388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8788110-186c-492f-a250-e414eb15184e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.172702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8788110-186c-492f-a250-e414eb15184e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.172968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8788110-186c-492f-a250-e414eb15184e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.185567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8788110-186c-492f-a250-e414eb15184e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d8788110-186c-492f-a250-e414eb15184e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:41.196179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:41.197293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 57/121] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:41.198406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8788110-186c-492f-a250-e414eb15184e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.201625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.235833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.264155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 65/122] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:41.264155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522414b-fda0-4061-8de2-523aee8bc94a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.264155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522414b-fda0-4061-8de2-523aee8bc94a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.264155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522414b-fda0-4061-8de2-523aee8bc94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.284874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a522414b-fda0-4061-8de2-523aee8bc94a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:41.286462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a522414b-fda0-4061-8de2-523aee8bc94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:41.342590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8788110-186c-492f-a250-e414eb15184e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.370253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 66/123] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:41.370253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522414b-fda0-4061-8de2-523aee8bc94a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.400510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.401538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.401962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.402457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.404441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.406167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.408363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 58/124] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.409357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.477115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.495179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522414b-fda0-4061-8de2-523aee8bc94a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.533555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.534435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.534843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.535210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.536129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:41.536555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.537809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.538244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.548680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 67/125] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.549548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.551866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b57193-3a2a-4177-939d-3f814f0e7914 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.592350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.593381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.593804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.594189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.595506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.597072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.604001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 59/126] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.604001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.704129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732d804-46c7-4856-9a40-ce2b5f884753 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.729949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064765d-9402-4442-b6ce-7546dcd37913 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.749558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.750724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.751123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.751513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.763772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:41.772526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:41.773655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:41.774685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:41.776314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.781546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 68/127] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:41.781546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.784335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.784795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.785239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.787409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.789630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.791902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.792833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 60/128] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.794163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.833973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:41.901688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137645a1-e418-40c4-bb6a-c6a51a08278a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.938579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43639e-dc0b-4b76-86db-5952d46b125f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:41.942134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.943153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.943546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.944014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.945423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.947178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:41.949142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 69/129] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:41.950415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:41.984354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:41.985992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:41.986455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:41.986888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:41.988149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] RBAC: Adding query filter params (role.id=01f210f8c6934006b4cb57b0fd1273bf, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=2be244ee2ef6473ba7ca7d3ca7edc3af) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:41.989014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:41.991223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.000213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 61/130] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/role_assignments?user.id=2be244ee2ef6473ba7ca7d3ca7edc3af&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=01f210f8c6934006b4cb57b0fd1273bf => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.001291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.091386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.124738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.148137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df44d02-d79b-4854-a4da-50e52061604e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.173702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ada4f-b2fc-4c62-a8d3-71a1d28770d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.205003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.206658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.209385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.209899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a/users/9936dac6582a4f55a12ec17571df8f3e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.216325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] RBAC: Authorizing `identity:create_grant(project_id=caaea6a33f9840079019e777b132555a, user_id=9936dac6582a4f55a12ec17571df8f3e, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.218479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.254206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 62/131] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:42 2022] PUT /identity/v3/projects/caaea6a33f9840079019e777b132555a/users/9936dac6582a4f55a12ec17571df8f3e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:42.256864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.278506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f73cce-9af5-43b8-8561-7ab36149f344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.279489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f73cce-9af5-43b8-8561-7ab36149f344 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.282084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f73cce-9af5-43b8-8561-7ab36149f344 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.282562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f73cce-9af5-43b8-8561-7ab36149f344 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.283563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f73cce-9af5-43b8-8561-7ab36149f344 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:42.283891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f73cce-9af5-43b8-8561-7ab36149f344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.285384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f73cce-9af5-43b8-8561-7ab36149f344 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.301397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 70/132] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.315379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b567a9-158e-4aa4-93d0-4bd737e814a1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.315749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b567a9-158e-4aa4-93d0-4bd737e814a1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.316061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b567a9-158e-4aa4-93d0-4bd737e814a1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.317788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 71/133] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:42.319003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b567a9-158e-4aa4-93d0-4bd737e814a1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.408731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004b5c9-d066-49b0-a1de-ae97978d5a85 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.446421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856801d-8c66-4940-970e-3aea2b1f0de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.448113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856801d-8c66-4940-970e-3aea2b1f0de5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.448868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856801d-8c66-4940-970e-3aea2b1f0de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.449692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856801d-8c66-4940-970e-3aea2b1f0de5 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.451699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856801d-8c66-4940-970e-3aea2b1f0de5 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.453471 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856801d-8c66-4940-970e-3aea2b1f0de5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.455940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 63/134] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.457784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7014d90-3519-426b-960c-5dd658542d49 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.458302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7014d90-3519-426b-960c-5dd658542d49 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.458789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7014d90-3519-426b-960c-5dd658542d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.476542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d7014d90-3519-426b-960c-5dd658542d49 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:42.478171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d7014d90-3519-426b-960c-5dd658542d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:42.499704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.519673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b567a9-158e-4aa4-93d0-4bd737e814a1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.566512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 64/135] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:42.567456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7014d90-3519-426b-960c-5dd658542d49 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.614342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.615533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.615837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.616124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.617690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.619258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.622417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 72/136] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:42.624848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.638019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.727346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7014d90-3519-426b-960c-5dd658542d49 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.784509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.787560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.788465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.789267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a/users/9936dac6582a4f55a12ec17571df8f3e/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.792648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=caaea6a33f9840079019e777b132555a, user_id=9936dac6582a4f55a12ec17571df8f3e, role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:42.795238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:42.817909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 65/137] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:42 2022] PUT /identity/v3/projects/caaea6a33f9840079019e777b132555a/users/9936dac6582a4f55a12ec17571df8f3e/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:42.820375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:42.821754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388490a-8433-4767-b256-0ed858d3463d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:42.850917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:42.912108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:42.918275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:42.918871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:42.919370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:42.939427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:42.958776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:42.959687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:42.960442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:42.965057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 73/138] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:42.972564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.096989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ac794-dded-4014-bce9-306f66f492f3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.145308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.189854 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.190767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.191067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.191362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] PATH_INFO: `/v3/projects/9172b93641c341519b19ef9fda02965e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.194667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] RBAC: Authorizing `identity:get_project(project_id=9172b93641c341519b19ef9fda02965e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.196256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.198234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 66/139] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/projects/9172b93641c341519b19ef9fda02965e => generated 312 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:43.205399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.221360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6011cdb-e272-4dd1-aac8-25ab13c9cedb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.267843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.268548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.268841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.269130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.269940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:43.270274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.272149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.282773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 74/140] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:43.289232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.421513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec45865-ca97-454e-a98c-3221914451fc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.457350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.467729 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.468645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.468931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.469221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] PATH_INFO: `/v3/users/fded449262ae4659b5d91d4b0bc2b7a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.470593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] RBAC: Authorizing `identity:get_user(user_id=fded449262ae4659b5d91d4b0bc2b7a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.472099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.473894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 75/141] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/users/fded449262ae4659b5d91d4b0bc2b7a0 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:43.478459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.485418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.530718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b1057-2772-4f56-99e8-d808cfacb43a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.605944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cf9be-fd53-4fe6-95db-2aa137ff1a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.607473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cf9be-fd53-4fe6-95db-2aa137ff1a55 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.607992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cf9be-fd53-4fe6-95db-2aa137ff1a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.608469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cf9be-fd53-4fe6-95db-2aa137ff1a55 None admin] PATH_INFO: `/v3/projects/9172b93641c341519b19ef9fda02965e/users/9936dac6582a4f55a12ec17571df8f3e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.616404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cf9be-fd53-4fe6-95db-2aa137ff1a55 None admin] RBAC: Authorizing `identity:create_grant(project_id=9172b93641c341519b19ef9fda02965e, user_id=9936dac6582a4f55a12ec17571df8f3e, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.618899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cf9be-fd53-4fe6-95db-2aa137ff1a55 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:43.636860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 67/142] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:43 2022] PUT /identity/v3/projects/9172b93641c341519b19ef9fda02965e/users/9936dac6582a4f55a12ec17571df8f3e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:43.638611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4d985-ff74-4919-9a04-574228ae7b2b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.639009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4d985-ff74-4919-9a04-574228ae7b2b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.639322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4d985-ff74-4919-9a04-574228ae7b2b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.640828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 68/143] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:43.647730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75be35-3894-42d2-ae77-c27998dc9cf7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.648051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75be35-3894-42d2-ae77-c27998dc9cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.648298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75be35-3894-42d2-ae77-c27998dc9cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.680114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6f75be35-3894-42d2-ae77-c27998dc9cf7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:43.681335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6f75be35-3894-42d2-ae77-c27998dc9cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:43.708452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8455c39-fa25-4b2e-a6ad-9d73f351267b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:43.788966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:43.789679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:43.790073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:43.790365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:43.881342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 69/144] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:43.898875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:43.909856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:43.912568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75be35-3894-42d2-ae77-c27998dc9cf7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:43.913327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.040178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 76/145] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:58:44.046647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.055238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.132147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75be35-3894-42d2-ae77-c27998dc9cf7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.174274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.183857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ec51d-8bf2-49cd-9486-8edb7c1107fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.189824 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.190848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.191282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.196633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.198235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.199878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.201775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 70/146] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:44.214293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.265334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.266653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.267340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.267791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.269409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.271142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.273075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 77/147] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:44.280640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:44.403446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e1a00-08f7-4caa-b631-ddd284303f81 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.447331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81baea-b4fa-4c84-aa4e-64a57fc47233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.448294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81baea-b4fa-4c84-aa4e-64a57fc47233 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.448724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81baea-b4fa-4c84-aa4e-64a57fc47233 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.449154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81baea-b4fa-4c84-aa4e-64a57fc47233 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.450083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81baea-b4fa-4c84-aa4e-64a57fc47233 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.452582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81baea-b4fa-4c84-aa4e-64a57fc47233 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.466003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:44.480431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 71/148] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:44 2022] POST /identity/v3/users => generated 283 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:58:44.520774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92853f43-339d-43fe-bd39-a0d7606c3bfd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:44.583596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e723642-4bdf-46a0-b869-7d169b262be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:44.584292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e723642-4bdf-46a0-b869-7d169b262be0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:44.584575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e723642-4bdf-46a0-b869-7d169b262be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:44.584848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e723642-4bdf-46a0-b869-7d169b262be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:44.585696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e723642-4bdf-46a0-b869-7d169b262be0 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=fded449262ae4659b5d91d4b0bc2b7a0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:44.586112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e723642-4bdf-46a0-b869-7d169b262be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:44.588043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e723642-4bdf-46a0-b869-7d169b262be0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:44.599452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 78/149] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:44 2022] GET /identity/v3/role_assignments?user.id=fded449262ae4659b5d91d4b0bc2b7a0&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:45.140810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2574-6282-4ff5-bdaf-35b93ddf29cf None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.141263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2574-6282-4ff5-bdaf-35b93ddf29cf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.141504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2574-6282-4ff5-bdaf-35b93ddf29cf None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.143167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 72/150] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:45.149054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194c15-aa62-47ba-b2f7-c28babc769e0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.149462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194c15-aa62-47ba-b2f7-c28babc769e0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.149669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06194c15-aa62-47ba-b2f7-c28babc769e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.168240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-06194c15-aa62-47ba-b2f7-c28babc769e0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:45.169454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-06194c15-aa62-47ba-b2f7-c28babc769e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:45.202140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.253487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 79/151] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:45.277401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2574-6282-4ff5-bdaf-35b93ddf29cf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.407614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2574-6282-4ff5-bdaf-35b93ddf29cf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.448805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731da8-dd49-481a-96d4-5bb7e7a89919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.449626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731da8-dd49-481a-96d4-5bb7e7a89919 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.450007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731da8-dd49-481a-96d4-5bb7e7a89919 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.450295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731da8-dd49-481a-96d4-5bb7e7a89919 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.462851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731da8-dd49-481a-96d4-5bb7e7a89919 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9e731da8-dd49-481a-96d4-5bb7e7a89919 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:45.471537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:45.472427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:45.473186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 73/152] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:45.480878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06194c15-aa62-47ba-b2f7-c28babc769e0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.609948 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06194c15-aa62-47ba-b2f7-c28babc769e0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.650339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.651178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.651467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.651628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06846b7b-2e33-4bbe-99f8-fdb5ee7f3f68 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.651777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.651919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06846b7b-2e33-4bbe-99f8-fdb5ee7f3f68 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.652173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06846b7b-2e33-4bbe-99f8-fdb5ee7f3f68 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.652500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:45.652810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:45.653698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 74/153] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:45.656147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.659617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e766b5-e899-4a21-84b6-97632d0e7f81 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.659852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e766b5-e899-4a21-84b6-97632d0e7f81 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.660132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e766b5-e899-4a21-84b6-97632d0e7f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.665879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 80/154] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:45.672566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.686825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d3e766b5-e899-4a21-84b6-97632d0e7f81 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:45.690549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d3e766b5-e899-4a21-84b6-97632d0e7f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:45.700995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.764818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:45.790956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3a4aa-3968-4e98-83cd-4df3db6aecaa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:45.808718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 75/155] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:45.837227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:45.837857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e766b5-e899-4a21-84b6-97632d0e7f81 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.838157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:45.838448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:45.838735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] PATH_INFO: `/v3/users/2be244ee2ef6473ba7ca7d3ca7edc3af` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:45.839991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] RBAC: Authorizing `identity:get_user(user_id=2be244ee2ef6473ba7ca7d3ca7edc3af)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:45.841514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:45.845358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 81/156] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/users/2be244ee2ef6473ba7ca7d3ca7edc3af => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:45.850947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:45.977211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e766b5-e899-4a21-84b6-97632d0e7f81 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.022211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.023380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.023857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.024365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.039411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:46.052719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:46.054157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:46.055018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:46.057503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 76/157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:46.064454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.120900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.125622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.151429 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f434224-e7f9-432d-ba6d-3944cdb5b392 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.228403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.229222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.229511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.229819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.232010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.236394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.238366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 82/158] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:45 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:46.247545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241adb3-860e-4832-a02f-59159727aab3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.265769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.304087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16563425-488e-4af6-9ee3-a7e1c7eb7ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.306160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563425-488e-4af6-9ee3-a7e1c7eb7ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.306467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563425-488e-4af6-9ee3-a7e1c7eb7ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.306756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563425-488e-4af6-9ee3-a7e1c7eb7ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.307594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16563425-488e-4af6-9ee3-a7e1c7eb7ab8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:46.307885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16563425-488e-4af6-9ee3-a7e1c7eb7ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.309291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16563425-488e-4af6-9ee3-a7e1c7eb7ab8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.330472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 77/159] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:46.330472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56337364-5605-40f1-a660-6caebf4293d1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.330472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56337364-5605-40f1-a660-6caebf4293d1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.330472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56337364-5605-40f1-a660-6caebf4293d1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.331183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 78/160] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:46.336843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfaab1-49e1-4e64-9a83-129ea400ad36 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.337134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfaab1-49e1-4e64-9a83-129ea400ad36 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.337406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfaab1-49e1-4e64-9a83-129ea400ad36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.394808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-98cfaab1-49e1-4e64-9a83-129ea400ad36 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:46.396025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-98cfaab1-49e1-4e64-9a83-129ea400ad36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:46.432089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c78daf-8127-4dd7-b493-9b3857810d8e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.495724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.496990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.497558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.498076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a/users/2be244ee2ef6473ba7ca7d3ca7edc3af/roles/01f210f8c6934006b4cb57b0fd1273bf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.501103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] RBAC: Authorizing `identity:create_grant(project_id=caaea6a33f9840079019e777b132555a, user_id=2be244ee2ef6473ba7ca7d3ca7edc3af, role_id=01f210f8c6934006b4cb57b0fd1273bf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.502832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.528452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 79/161] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:46.530086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfaab1-49e1-4e64-9a83-129ea400ad36 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.535693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 83/162] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:46 2022] PUT /identity/v3/projects/caaea6a33f9840079019e777b132555a/users/2be244ee2ef6473ba7ca7d3ca7edc3af/roles/01f210f8c6934006b4cb57b0fd1273bf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:58:46.564896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.585497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.599772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.714872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfaab1-49e1-4e64-9a83-129ea400ad36 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.786361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.787982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.788411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.788830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.791649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:46.794200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:46.797586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 80/163] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:46.803573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.821753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba29b8b-a809-4c0f-a5fc-46d489c249ab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.899347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.900302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.900593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.900970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:46.930793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:46.942607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:46.943531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:46.944330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:46.944920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527add2f-0112-4494-a53b-9eb1357cc897 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:46.944997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 84/164] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:46.953130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:46.962209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:46.988786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:46.989463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:46.989749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:46.990180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.110763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.112609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.117594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.195529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301e558-7fa7-4448-b1b8-ff3f68a183b4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.273223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824db79-bf6d-41ba-97e6-d04734be4d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.273976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824db79-bf6d-41ba-97e6-d04734be4d13 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.274271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824db79-bf6d-41ba-97e6-d04734be4d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.274558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824db79-bf6d-41ba-97e6-d04734be4d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.275387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824db79-bf6d-41ba-97e6-d04734be4d13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:47.275744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824db79-bf6d-41ba-97e6-d04734be4d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.277734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824db79-bf6d-41ba-97e6-d04734be4d13 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.297537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 85/165] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:47.298933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d2fa0-d495-4619-8663-4979f44b8b05 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.299337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d2fa0-d495-4619-8663-4979f44b8b05 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.299642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d2fa0-d495-4619-8663-4979f44b8b05 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.301086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 86/166] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:47.306376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b00b5-cd61-4229-a565-aa913b4b12ec None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.306806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b00b5-cd61-4229-a565-aa913b4b12ec None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.307133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b00b5-cd61-4229-a565-aa913b4b12ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.335653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9e6b00b5-cd61-4229-a565-aa913b4b12ec None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:47.336868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9e6b00b5-cd61-4229-a565-aa913b4b12ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:47.367444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 81/167] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:47.368223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.374642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.520658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979eb402-c582-46f4-9b5f-ed3df805b5cb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.528194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 87/168] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:47.529159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b00b5-cd61-4229-a565-aa913b4b12ec None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.584577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.585487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.585830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.586118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.587400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.589928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.591913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 82/169] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:47.592755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.679910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.682819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:47.725368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229cb2-f660-4e9d-91f8-210ba659f0b8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.747963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b00b5-cd61-4229-a565-aa913b4b12ec None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:47.799561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.800929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.801455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.801971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.820469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:47.824144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:47.824720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:47.825467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:47.826240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:47.839019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:47.842394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:47.844975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 88/170] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:47.846097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:47.847223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:47.848093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:47.848889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:47.856141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 83/171] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:47.859229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.141343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731e3d9-da29-4db0-900e-85880d535af2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.164045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397f5e2-39c5-4cf8-ac20-4e8c730afaca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.258982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.264976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.265775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.266194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.266459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.273883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.275330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.276094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.276872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.278836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=9936dac6582a4f55a12ec17571df8f3e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:48.279753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.281080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.283177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.294958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 84/172] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/role_assignments?user.id=9936dac6582a4f55a12ec17571df8f3e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 608 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:48.297258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.414577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.416624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.463112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e2bbb-4fb0-4cd3-bf01-3af6267d58da None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.525064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.525780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.526255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.526698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.527601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:48.528050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.529587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.545281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 85/173] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:48.551294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.617093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 89/174] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 771 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:48.623233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.658199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.678159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.717137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de344b-729c-48ef-91e4-942945686fbf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.754149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6add6-36d7-46eb-9cef-8e8a036677d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.786481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7f4a0-b566-40b1-aff1-0dc1bfde2721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.787629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7f4a0-b566-40b1-aff1-0dc1bfde2721 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.788051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7f4a0-b566-40b1-aff1-0dc1bfde2721 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.788494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7f4a0-b566-40b1-aff1-0dc1bfde2721 None admin] PATH_INFO: `/v3/users/9936dac6582a4f55a12ec17571df8f3e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.789936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7f4a0-b566-40b1-aff1-0dc1bfde2721 None admin] RBAC: Authorizing `identity:get_user(user_id=9936dac6582a4f55a12ec17571df8f3e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.791528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7f4a0-b566-40b1-aff1-0dc1bfde2721 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.792916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.794044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.794502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.794614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 86/175] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/users/9936dac6582a4f55a12ec17571df8f3e => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:48.795015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.796487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.797148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcdc26-5e39-4a98-a0e0-4c8441ac44a6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.797431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcdc26-5e39-4a98-a0e0-4c8441ac44a6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.797708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcdc26-5e39-4a98-a0e0-4c8441ac44a6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.798580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.799035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 87/176] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:48.802621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 90/177] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:48.805390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981f82-b048-4e5a-b0c7-cc947064c5b1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.805895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981f82-b048-4e5a-b0c7-cc947064c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.806351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d981f82-b048-4e5a-b0c7-cc947064c5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.808825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.843349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0d981f82-b048-4e5a-b0c7-cc947064c5b1 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:48.845018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0d981f82-b048-4e5a-b0c7-cc947064c5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:48.940519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3008a3d-ae6b-4c2e-85e6-4bb190623ef9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:48.943208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:48.952412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 88/178] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:48.953550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d981f82-b048-4e5a-b0c7-cc947064c5b1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:48.980493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:48.981493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:48.981971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:48.982380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:48.983889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] RBAC: Adding query filter params (role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=9936dac6582a4f55a12ec17571df8f3e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:48.984372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:48.986015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:48.992709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 91/179] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/role_assignments?user.id=9936dac6582a4f55a12ec17571df8f3e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:49.003342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.021243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.076312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d981f82-b048-4e5a-b0c7-cc947064c5b1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.121839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a20ad6-f659-4cb8-9cae-15630ace7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.122991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20ad6-f659-4cb8-9cae-15630ace7d59 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.123517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20ad6-f659-4cb8-9cae-15630ace7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.124054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20ad6-f659-4cb8-9cae-15630ace7d59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.204424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2672c7-870e-439d-a53c-9245fd400432 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.214657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a20ad6-f659-4cb8-9cae-15630ace7d59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.218039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a20ad6-f659-4cb8-9cae-15630ace7d59 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.287149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.287987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.288444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.288870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.302312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.312660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:49.328835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:49.329824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:49.330618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:49.334215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 92/180] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:49.343300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.356174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 89/181] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:49.358901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46504a0e-eeeb-49ca-8024-328aaecabb78 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.359253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46504a0e-eeeb-49ca-8024-328aaecabb78 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.359543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46504a0e-eeeb-49ca-8024-328aaecabb78 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.361104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 90/182] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:49.365895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46504a0e-eeeb-49ca-8024-328aaecabb78 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.400861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.506549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46504a0e-eeeb-49ca-8024-328aaecabb78 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.519147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad707e-c443-482e-b47c-c6c8bf3da4c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.547284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ea618-cd87-462b-bb47-0cd3b0c08d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.548229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ea618-cd87-462b-bb47-0cd3b0c08d99 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.548506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ea618-cd87-462b-bb47-0cd3b0c08d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.549021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ea618-cd87-462b-bb47-0cd3b0c08d99 None admin] PATH_INFO: `/v3/projects/9172b93641c341519b19ef9fda02965e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.550637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ea618-cd87-462b-bb47-0cd3b0c08d99 None admin] RBAC: Authorizing `identity:get_project(project_id=9172b93641c341519b19ef9fda02965e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.552238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ea618-cd87-462b-bb47-0cd3b0c08d99 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.555291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 91/183] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/projects/9172b93641c341519b19ef9fda02965e => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.557088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56847f63-671e-4055-94b4-8a01868beaf7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.557611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56847f63-671e-4055-94b4-8a01868beaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.558346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56847f63-671e-4055-94b4-8a01868beaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.564057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.565025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.565448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.565897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.566886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.567367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.568939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.579707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 93/184] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.579888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-56847f63-671e-4055-94b4-8a01868beaf7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:49.580998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.581133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-56847f63-671e-4055-94b4-8a01868beaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:49.603580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.659199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 92/185] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:49.660478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56847f63-671e-4055-94b4-8a01868beaf7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.688932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.701342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b5917-55b2-4cf8-b83a-8bc058c69108 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.741221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.742268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.742749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.743207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.744299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=9172b93641c341519b19ef9fda02965e, user.id=9936dac6582a4f55a12ec17571df8f3e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:49.744967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.746910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.754564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 94/186] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/role_assignments?user.id=9936dac6582a4f55a12ec17571df8f3e&scope.project.id=9172b93641c341519b19ef9fda02965e&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.763612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.770347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56847f63-671e-4055-94b4-8a01868beaf7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.814604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.815852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.816349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.816894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] PATH_INFO: `/v3/users/09753d93ba6b431b93ec797491cf51aa` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.859284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] RBAC: Authorizing `identity:get_user(user_id=09753d93ba6b431b93ec797491cf51aa)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:49.861095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:49.863246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 93/187] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/users/09753d93ba6b431b93ec797491cf51aa => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:49.869440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:49.904535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695fc2f-a520-487b-85d4-a47b85e8273d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:49.925197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:49.950800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:49.952414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:49.952837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:49.953271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:49.967561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:49.976550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:49.978791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:49.979906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:49.982501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 95/188] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:49.990887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.031644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.049288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f254a5a-b24d-4f95-82aa-3d57d340e629 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.098680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c83c-434d-4b71-82be-87eafec15101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.099352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c83c-434d-4b71-82be-87eafec15101 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.099640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c83c-434d-4b71-82be-87eafec15101 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.099917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c83c-434d-4b71-82be-87eafec15101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.154956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c0244-a356-42e0-ba2f-eca083472748 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.197105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c83c-434d-4b71-82be-87eafec15101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.199146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c83c-434d-4b71-82be-87eafec15101 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.202127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff05f7-341a-40a1-a807-c1454d67070d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.203325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff05f7-341a-40a1-a807-c1454d67070d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.204771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff05f7-341a-40a1-a807-c1454d67070d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.204771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff05f7-341a-40a1-a807-c1454d67070d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.205036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff05f7-341a-40a1-a807-c1454d67070d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:50.205357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff05f7-341a-40a1-a807-c1454d67070d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.206856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbff05f7-341a-40a1-a807-c1454d67070d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.212439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.216389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 96/189] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.217966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338940c-21e2-48c6-a34d-4e94266b86f6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.218262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338940c-21e2-48c6-a34d-4e94266b86f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.218708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338940c-21e2-48c6-a34d-4e94266b86f6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.221756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 97/190] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:50.225931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338940c-21e2-48c6-a34d-4e94266b86f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.320756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.337355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 94/191] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:58:50.338978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616bed-ff3c-409b-a5a6-808fbf4e1d47 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.339267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616bed-ff3c-409b-a5a6-808fbf4e1d47 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.339549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4616bed-ff3c-409b-a5a6-808fbf4e1d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.350163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338940c-21e2-48c6-a34d-4e94266b86f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.356822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e4616bed-ff3c-409b-a5a6-808fbf4e1d47 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:50.358039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e4616bed-ff3c-409b-a5a6-808fbf4e1d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:50.387012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d2fc4-ad1d-4358-8aa1-50212a244f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.388426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d2fc4-ad1d-4358-8aa1-50212a244f9e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.388910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d2fc4-ad1d-4358-8aa1-50212a244f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.389382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d2fc4-ad1d-4358-8aa1-50212a244f9e None admin] PATH_INFO: `/v3/users/09753d93ba6b431b93ec797491cf51aa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.390919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d2fc4-ad1d-4358-8aa1-50212a244f9e None admin] RBAC: Authorizing `identity:get_user(user_id=09753d93ba6b431b93ec797491cf51aa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.392709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d2fc4-ad1d-4358-8aa1-50212a244f9e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.394880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 98/192] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/users/09753d93ba6b431b93ec797491cf51aa => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:50.396651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488c18-29c1-4a75-965e-8fcf15cc35f0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.397050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488c18-29c1-4a75-965e-8fcf15cc35f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.397359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77488c18-29c1-4a75-965e-8fcf15cc35f0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.398838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 99/193] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:50.399645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488c18-29c1-4a75-965e-8fcf15cc35f0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.432296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 95/194] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:50.441944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4616bed-ff3c-409b-a5a6-808fbf4e1d47 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.483713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.509326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77488c18-29c1-4a75-965e-8fcf15cc35f0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.542880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4616bed-ff3c-409b-a5a6-808fbf4e1d47 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.550228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fa334-456e-4594-94d3-0c560e3c40ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.551111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fa334-456e-4594-94d3-0c560e3c40ff None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.551393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fa334-456e-4594-94d3-0c560e3c40ff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.551668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fa334-456e-4594-94d3-0c560e3c40ff None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.552883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fa334-456e-4594-94d3-0c560e3c40ff None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.554369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fa334-456e-4594-94d3-0c560e3c40ff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.556058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 100/195] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.557261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b4e87-87c5-4f99-9b2e-2646e94040b0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.557548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b4e87-87c5-4f99-9b2e-2646e94040b0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.557853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b4e87-87c5-4f99-9b2e-2646e94040b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.575176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7e7b4e87-87c5-4f99-9b2e-2646e94040b0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:50.576302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e7b4e87-87c5-4f99-9b2e-2646e94040b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:50.586241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.586734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.586895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.587177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.587450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.652106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 101/196] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:50.652970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b4e87-87c5-4f99-9b2e-2646e94040b0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.657329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.659036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.703613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.766494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 96/197] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:50.767344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.775521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b4e87-87c5-4f99-9b2e-2646e94040b0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.810813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.811646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.811940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.812216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.813384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.814896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.816523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 102/198] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.817270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.832541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.876188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5876a-f25e-4bee-9f46-41ae2b080530 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.917022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.917676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.917997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.918275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.919182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=09753d93ba6b431b93ec797491cf51aa) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:50.919506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:50.921510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:50.927410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 97/199] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/role_assignments?user.id=09753d93ba6b431b93ec797491cf51aa&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:50.928008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:50.928663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118dcec-42dc-428d-9510-e402644279c0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:50.943150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:50.974286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:50.975102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:50.975414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:50.975692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:50.989002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:50.997044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:50.998173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:50.999516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 103/200] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:51.000217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.069039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda512b0-9924-45af-8850-586d3da7c63c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.084299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.108016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.109405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.109876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.110161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.111389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.112925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.114794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 98/201] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:50 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.120002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.120731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bc19a-918d-463f-8dbf-6690ea4ea21f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.161768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.162728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.163215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.163666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.164549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.166613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.197645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 104/202] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:50 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:51.198957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.211321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.258390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ecdd-55d4-4fd1-a63c-01b833c1089d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.310253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.311625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.311979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.312523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.313310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:51.313627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.315793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.327819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 99/203] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.333791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.337782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c01998-959c-4561-abe2-2f11572ae893 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.370968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.382434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.383664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.384332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.385178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.386093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] RBAC: Adding query filter params (role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=09753d93ba6b431b93ec797491cf51aa) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:51.386666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.388673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.396360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 105/204] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/role_assignments?user.id=09753d93ba6b431b93ec797491cf51aa&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.463614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b4db-0031-4746-bc27-e54b9d2ee142 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.503618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.504528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.504815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.505107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] PATH_INFO: `/v3/users/2be244ee2ef6473ba7ca7d3ca7edc3af` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.506470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] RBAC: Authorizing `identity:get_user(user_id=2be244ee2ef6473ba7ca7d3ca7edc3af)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.507984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.509796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 100/205] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/users/2be244ee2ef6473ba7ca7d3ca7edc3af => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:51.516170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.643471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07c57f-2777-4b53-aec2-e2db2b4e55fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:51.683614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:51.684578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:51.685167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:51.685842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:51.765242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:51.770436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:51.772738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:51.883500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 106/206] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:51.889138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:51.892681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.046821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998bb7e-3cd0-4e66-8a00-bec220738139 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.100570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812fa97-1d99-45d2-ae07-2de72424d0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.101497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812fa97-1d99-45d2-ae07-2de72424d0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.101816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812fa97-1d99-45d2-ae07-2de72424d0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.102103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812fa97-1d99-45d2-ae07-2de72424d0d3 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.103526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812fa97-1d99-45d2-ae07-2de72424d0d3 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.105034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812fa97-1d99-45d2-ae07-2de72424d0d3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.107260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 101/207] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:51 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:52.113739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:52.235580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b80ef4-4f26-4816-b1f6-1462e468591b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:52.244266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:52.273742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a70a7-294d-40b9-846e-feb1e09cb154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:52.274584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a70a7-294d-40b9-846e-feb1e09cb154 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:52.274997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a70a7-294d-40b9-846e-feb1e09cb154 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:52.275379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a70a7-294d-40b9-846e-feb1e09cb154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:52.276332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a70a7-294d-40b9-846e-feb1e09cb154 None admin] RBAC: Adding query filter params (role.id=01f210f8c6934006b4cb57b0fd1273bf, scope.project.id=caaea6a33f9840079019e777b132555a, user.id=2be244ee2ef6473ba7ca7d3ca7edc3af) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:52.276757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a70a7-294d-40b9-846e-feb1e09cb154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:52.278364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a70a7-294d-40b9-846e-feb1e09cb154 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:52.284989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 107/208] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:52 2022] GET /identity/v3/role_assignments?user.id=2be244ee2ef6473ba7ca7d3ca7edc3af&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=01f210f8c6934006b4cb57b0fd1273bf => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:53.336065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c62b8-4cdf-4bd7-aa68-3ffd989d0e73 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.336572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c62b8-4cdf-4bd7-aa68-3ffd989d0e73 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.336647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c62b8-4cdf-4bd7-aa68-3ffd989d0e73 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.338328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 102/209] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:53.344006 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e4756-a20c-49ae-8f90-3f7d7e5e7b23 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.344407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e4756-a20c-49ae-8f90-3f7d7e5e7b23 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.344846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e4756-a20c-49ae-8f90-3f7d7e5e7b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.364135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-747e4756-a20c-49ae-8f90-3f7d7e5e7b23 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:53.365313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-747e4756-a20c-49ae-8f90-3f7d7e5e7b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:53.457951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 108/210] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:53.484296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c62b8-4cdf-4bd7-aa68-3ffd989d0e73 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.571883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.610297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c62b8-4cdf-4bd7-aa68-3ffd989d0e73 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.649877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.650788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.651073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.651351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.663236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:53.671030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:53.672892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:53.673802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 103/211] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:53.678917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e4756-a20c-49ae-8f90-3f7d7e5e7b23 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.777567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:53.795697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e4756-a20c-49ae-8f90-3f7d7e5e7b23 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:53.845127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92222666-c4a8-46ea-a065-37aa3957fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:53.846179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92222666-c4a8-46ea-a065-37aa3957fa25 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.846489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92222666-c4a8-46ea-a065-37aa3957fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.846818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92222666-c4a8-46ea-a065-37aa3957fa25 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.847844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92222666-c4a8-46ea-a065-37aa3957fa25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:53.848247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92222666-c4a8-46ea-a065-37aa3957fa25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:53.850059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92222666-c4a8-46ea-a065-37aa3957fa25 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:53.860587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 109/212] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:53.866778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:53.901993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2c46d-0abf-461a-bed8-8fda01df48fb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.902434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2c46d-0abf-461a-bed8-8fda01df48fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.902718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2c46d-0abf-461a-bed8-8fda01df48fb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.905689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 110/213] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:53.907636 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07a4f-ec2e-4131-a05f-16a34c5a5829 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.907909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07a4f-ec2e-4131-a05f-16a34c5a5829 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.908190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad07a4f-ec2e-4131-a05f-16a34c5a5829 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.909595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 111/214] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:53.910665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708e2a-c428-4fd0-be57-11ba42eb0ff8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:53.911013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708e2a-c428-4fd0-be57-11ba42eb0ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:53.911296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db708e2a-c428-4fd0-be57-11ba42eb0ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:53.928696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-db708e2a-c428-4fd0-be57-11ba42eb0ff8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:53.929940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db708e2a-c428-4fd0-be57-11ba42eb0ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:54.003649 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 112/215] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:54.004915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee096e-3745-489e-bfcf-08d1a99a468a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.005303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeee836-c791-4949-8461-4562e51e20ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.005666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeee836-c791-4949-8461-4562e51e20ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.006095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeee836-c791-4949-8461-4562e51e20ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.025820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-baeee836-c791-4949-8461-4562e51e20ca None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:54.027555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-baeee836-c791-4949-8461-4562e51e20ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:54.046091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.053239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.054268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.054538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.054816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] PATH_INFO: `/v3/users/09753d93ba6b431b93ec797491cf51aa` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.056006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] RBAC: Authorizing `identity:get_user(user_id=09753d93ba6b431b93ec797491cf51aa)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.057848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.061944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 104/216] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:53 2022] GET /identity/v3/users/09753d93ba6b431b93ec797491cf51aa => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.064845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.112668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 113/217] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:54.123018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeee836-c791-4949-8461-4562e51e20ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.142344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.213161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e36bbd-4be2-4157-8b80-b40512058cfc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.248333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.249145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.249453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.249761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.260035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:54.261327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeee836-c791-4949-8461-4562e51e20ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:54.268570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:54.269631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:54.270469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 105/218] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:54.271136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.300942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f63294-f75c-4c72-9bd4-4d25a43956e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.301751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f63294-f75c-4c72-9bd4-4d25a43956e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.302041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f63294-f75c-4c72-9bd4-4d25a43956e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.302315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f63294-f75c-4c72-9bd4-4d25a43956e6 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.303479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f63294-f75c-4c72-9bd4-4d25a43956e6 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.304937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f63294-f75c-4c72-9bd4-4d25a43956e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.306658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 114/219] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.307801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42477a0-d4e1-49b8-8407-098a65b7069f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.308082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42477a0-d4e1-49b8-8407-098a65b7069f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.308356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42477a0-d4e1-49b8-8407-098a65b7069f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.309742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 115/220] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:54.310278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42477a0-d4e1-49b8-8407-098a65b7069f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.331329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.375315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c20f169-1a66-4297-be36-80d1a1e3ef82 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.387311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.410658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.411449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.411738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.412018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.414312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42477a0-d4e1-49b8-8407-098a65b7069f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.422236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:54.430155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:54.431019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:54.431855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 106/221] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:54.432510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.450711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab981a-f7f7-4da0-b09d-784b87c25777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.451390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab981a-f7f7-4da0-b09d-784b87c25777 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.451673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab981a-f7f7-4da0-b09d-784b87c25777 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.451956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab981a-f7f7-4da0-b09d-784b87c25777 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.453197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab981a-f7f7-4da0-b09d-784b87c25777 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:54.453508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab981a-f7f7-4da0-b09d-784b87c25777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.454922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab981a-f7f7-4da0-b09d-784b87c25777 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.462995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 116/222] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.464167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e08eea-abdf-4362-889a-4d320721cadc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.464458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e08eea-abdf-4362-889a-4d320721cadc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.464737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e08eea-abdf-4362-889a-4d320721cadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.480934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7e08eea-abdf-4362-889a-4d320721cadc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:54.482147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7e08eea-abdf-4362-889a-4d320721cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:54.533976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401eecc-49bf-4c75-8108-8b945f684889 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.547441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 117/223] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:54.548347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e08eea-abdf-4362-889a-4d320721cadc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.555467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.577306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.578771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.579056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.579333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/09753d93ba6b431b93ec797491cf51aa/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.581298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=51d1f730296e4ce69df77b2c3a336fc3, user_id=09753d93ba6b431b93ec797491cf51aa, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.582784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.595927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 107/224] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:54 2022] PUT /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/09753d93ba6b431b93ec797491cf51aa/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:54.609962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.623087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.662531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e08eea-abdf-4362-889a-4d320721cadc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.702160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.703440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.703725 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.704004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.704768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:54.705077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.706534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.712680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2feb1-bfae-4e40-bca9-59921de18b3f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.716270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 118/225] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.716939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.759615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.760817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.761280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.761749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] PATH_INFO: `/v3/users/09753d93ba6b431b93ec797491cf51aa` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.763219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] RBAC: Authorizing `identity:get_user(user_id=09753d93ba6b431b93ec797491cf51aa)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.764844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.767800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 108/226] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/users/09753d93ba6b431b93ec797491cf51aa => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.769266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.825536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.831545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8ce1c-fb33-4d00-aef5-fc061acaa5c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.866878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:54.877552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.878650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.879098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.879389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.881667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.883137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.884864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 119/227] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.884864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:54.894054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599baec-68b6-4f6f-9d09-e1051eb67548 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:54.931353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:54.932533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:54.932966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:54.933388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] PATH_INFO: `/v3/users/5ae1201438b246e0a895d8042e634e23` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:54.949156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] RBAC: Authorizing `identity:get_user(user_id=5ae1201438b246e0a895d8042e634e23)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:54.950939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:54.953009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 109/228] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/users/5ae1201438b246e0a895d8042e634e23 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:54.953782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.021711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cea83-516f-47ec-94cf-90e78b20d0a8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.066153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.067025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.067311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.069306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.069306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.070307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.077866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 120/229] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:54 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:55.077866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.089367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd641a-b3e2-4fad-b7cf-13fb68f0ac2f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.092581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.130327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.130843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.133760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.133760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.133760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.133760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.157345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.169748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 110/230] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:54 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:55.169748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.223212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b634009-f989-4ea3-904b-73d54e4ec187 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.265180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.266427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.266717 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.267001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.289638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323996-b4ec-4b8f-8eb3-ef2169ecbc7b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.328797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.330700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.330700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.330700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/09753d93ba6b431b93ec797491cf51aa/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.332741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] RBAC: Authorizing `identity:create_grant(project_id=51d1f730296e4ce69df77b2c3a336fc3, user_id=09753d93ba6b431b93ec797491cf51aa, role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.334546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.348842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.349795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 111/231] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:58:55 2022] PUT /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/09753d93ba6b431b93ec797491cf51aa/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:58:55.350383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.430040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.476185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 121/232] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:55.482042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.511389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:55.608705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd434e-cc74-4e54-b1cb-e16f1ce97765 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.643933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31522eeb-0ab2-4e42-a4c1-2a0b9519d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.644814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31522eeb-0ab2-4e42-a4c1-2a0b9519d006 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.645093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31522eeb-0ab2-4e42-a4c1-2a0b9519d006 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.645364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31522eeb-0ab2-4e42-a4c1-2a0b9519d006 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.646582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31522eeb-0ab2-4e42-a4c1-2a0b9519d006 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.648026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31522eeb-0ab2-4e42-a4c1-2a0b9519d006 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.653875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 112/233] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:55 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:58:55.656618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:55.776914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8efb43-78f6-4450-9123-98492a06066e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:55.824784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac0fc1-4cf2-4810-ac1e-716eb3892085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:55.825832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac0fc1-4cf2-4810-ac1e-716eb3892085 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:55.826344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac0fc1-4cf2-4810-ac1e-716eb3892085 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:55.826822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac0fc1-4cf2-4810-ac1e-716eb3892085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:55.828377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac0fc1-4cf2-4810-ac1e-716eb3892085 None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=5ae1201438b246e0a895d8042e634e23) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:55.828891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac0fc1-4cf2-4810-ac1e-716eb3892085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:55.830588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac0fc1-4cf2-4810-ac1e-716eb3892085 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:55.837615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 122/234] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:55 2022] GET /identity/v3/role_assignments?user.id=5ae1201438b246e0a895d8042e634e23&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:57.076513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35426657-80ae-49fc-b9d4-0fb6705312e1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.077021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35426657-80ae-49fc-b9d4-0fb6705312e1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.077095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35426657-80ae-49fc-b9d4-0fb6705312e1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.078692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 113/235] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:57.084337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8aace-18e4-4d70-b4a2-0cf9a6d3bfe6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.084740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8aace-18e4-4d70-b4a2-0cf9a6d3bfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.084906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8aace-18e4-4d70-b4a2-0cf9a6d3bfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.106041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-46a8aace-18e4-4d70-b4a2-0cf9a6d3bfe6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:57.107239 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-46a8aace-18e4-4d70-b4a2-0cf9a6d3bfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:57.144496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:57.208798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 123/236] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:57.258212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35426657-80ae-49fc-b9d4-0fb6705312e1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.366477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:57.386140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35426657-80ae-49fc-b9d4-0fb6705312e1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:57.426114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:57.426916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.427200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.427480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.438167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:57.446140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:57.447085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:57.447917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 114/237] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:57.455014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8aace-18e4-4d70-b4a2-0cf9a6d3bfe6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.597069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8aace-18e4-4d70-b4a2-0cf9a6d3bfe6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:57.646632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:57.647239 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.647528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.647807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.648562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:57.648870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:57.650303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:57.666016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 124/238] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:57.672570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.853821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6716b74-124b-456b-8bcf-f99a70d71627 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:57.911585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe61d13-5369-42fc-8e6b-b243f2250630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:57.912860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe61d13-5369-42fc-8e6b-b243f2250630 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.913352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe61d13-5369-42fc-8e6b-b243f2250630 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.913883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe61d13-5369-42fc-8e6b-b243f2250630 None admin] PATH_INFO: `/v3/users/09753d93ba6b431b93ec797491cf51aa` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.915629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe61d13-5369-42fc-8e6b-b243f2250630 None admin] RBAC: Authorizing `identity:get_user(user_id=09753d93ba6b431b93ec797491cf51aa)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:57.917405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe61d13-5369-42fc-8e6b-b243f2250630 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:57.919683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 115/239] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/users/09753d93ba6b431b93ec797491cf51aa => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:57.926315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:57.956029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:57.994733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b638-e12e-42de-ba6d-74ccbc111744 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:57.995083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b638-e12e-42de-ba6d-74ccbc111744 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:57.995305 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881b638-e12e-42de-ba6d-74ccbc111744 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:57.996751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 116/240] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:58:58.003319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe9f3d-2ac5-4331-b278-6f09607b69d9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.003628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe9f3d-2ac5-4331-b278-6f09607b69d9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.003881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe9f3d-2ac5-4331-b278-6f09607b69d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.029199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9bfe9f3d-2ac5-4331-b278-6f09607b69d9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:58.030460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9bfe9f3d-2ac5-4331-b278-6f09607b69d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:58.058058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.115992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63953940-ea7e-4fd4-8ece-e514506bb98c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.153476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 117/241] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:58:58.155334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1117e-44f7-4fcf-8da6-d8babb0b1ded None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.155626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1117e-44f7-4fcf-8da6-d8babb0b1ded None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.155907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1117e-44f7-4fcf-8da6-d8babb0b1ded None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.157350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 118/242] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:58.164366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35827608-e1fa-45b2-8190-f341f38a1c75 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.164614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35827608-e1fa-45b2-8190-f341f38a1c75 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.164856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35827608-e1fa-45b2-8190-f341f38a1c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.167693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca149397-133d-49e9-88e7-9079674ecb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.168361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149397-133d-49e9-88e7-9079674ecb57 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.168647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149397-133d-49e9-88e7-9079674ecb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.168924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149397-133d-49e9-88e7-9079674ecb57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.193397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-35827608-e1fa-45b2-8190-f341f38a1c75 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:58.194983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-35827608-e1fa-45b2-8190-f341f38a1c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:58.252042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca149397-133d-49e9-88e7-9079674ecb57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.253551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca149397-133d-49e9-88e7-9079674ecb57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.276014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 119/243] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:58.277197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35827608-e1fa-45b2-8190-f341f38a1c75 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.283216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.358289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.383813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35827608-e1fa-45b2-8190-f341f38a1c75 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.388172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 125/244] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:58.389609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fb650-166d-42c4-9d39-b8553363a3b7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.390225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fb650-166d-42c4-9d39-b8553363a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.391347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fb650-166d-42c4-9d39-b8553363a3b7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.397105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 126/245] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:58.397105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fb650-166d-42c4-9d39-b8553363a3b7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.431671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.432412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.432694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.432969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.445588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:58.454974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:58.455906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:58.456741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 120/246] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:58.457548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.520611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fb650-166d-42c4-9d39-b8553363a3b7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.548919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.565799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03273ebe-1f77-4c33-aac2-a026b5530f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.566631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03273ebe-1f77-4c33-aac2-a026b5530f41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.566918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03273ebe-1f77-4c33-aac2-a026b5530f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.567193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03273ebe-1f77-4c33-aac2-a026b5530f41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.569642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03273ebe-1f77-4c33-aac2-a026b5530f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.571207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03273ebe-1f77-4c33-aac2-a026b5530f41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.573225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 127/247] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:58.576780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0794d-18e6-4f4a-bf35-aa3e97399902 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.577356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0794d-18e6-4f4a-bf35-aa3e97399902 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.577855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0794d-18e6-4f4a-bf35-aa3e97399902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.599656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa54063-922a-403b-9ea5-206a45c6a786 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.603612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3ca0794d-18e6-4f4a-bf35-aa3e97399902 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:58.604834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ca0794d-18e6-4f4a-bf35-aa3e97399902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:58.646360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.647228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.647508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.647786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.649505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.651064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.652833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 121/248] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:58.653603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.658026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.685000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 128/249] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:58.686195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0794d-18e6-4f4a-bf35-aa3e97399902 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.765218 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14193e6f-99bb-45b3-a306-dc24cc3c0208 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.798576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0794d-18e6-4f4a-bf35-aa3e97399902 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:58.800801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca841c-d669-41ba-a6d6-f38207509886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.801476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca841c-d669-41ba-a6d6-f38207509886 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.801782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca841c-d669-41ba-a6d6-f38207509886 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.802065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca841c-d669-41ba-a6d6-f38207509886 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.802819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca841c-d669-41ba-a6d6-f38207509886 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:58.803127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca841c-d669-41ba-a6d6-f38207509886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.804541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca841c-d669-41ba-a6d6-f38207509886 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.813370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 122/250] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:58.815293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc4972-a043-4a50-a42a-44b44de6dbf5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.815582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc4972-a043-4a50-a42a-44b44de6dbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.815860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc4972-a043-4a50-a42a-44b44de6dbf5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.818705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.821627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 123/251] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:58:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:58:58.822547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc4972-a043-4a50-a42a-44b44de6dbf5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.839428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:58.840176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:58.840561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:58.840927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:58.841740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:58.843319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:58.866980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 129/252] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:58:58 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:58:58.867829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:58.916309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:58.973381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc4972-a043-4a50-a42a-44b44de6dbf5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.009993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0b334-be02-49fd-aa74-a0bfeec47ef9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.019789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.020485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.020767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.021046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.021936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=09753d93ba6b431b93ec797491cf51aa) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:59.022331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.023865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.041752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 124/253] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/role_assignments?user.id=09753d93ba6b431b93ec797491cf51aa&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:59.041752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.065932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa5b42-3c21-451e-b8be-64a3add95f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.067255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5b42-3c21-451e-b8be-64a3add95f12 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.067551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5b42-3c21-451e-b8be-64a3add95f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.067835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa5b42-3c21-451e-b8be-64a3add95f12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.088631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa5b42-3c21-451e-b8be-64a3add95f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2cfa5b42-3c21-451e-b8be-64a3add95f12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:59.098501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:59.099680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:58:59.100671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 130/254] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:58:59.101470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14dc0-3d93-4280-817b-08c348219435 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.101710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14dc0-3d93-4280-817b-08c348219435 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.102017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14dc0-3d93-4280-817b-08c348219435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.127794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-60a14dc0-3d93-4280-817b-08c348219435 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:58:59.129033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-60a14dc0-3d93-4280-817b-08c348219435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:58:59.174804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.205578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee85e-f1e0-43f3-bb7c-fd6257dc6876 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.236881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 131/255] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:58:59.238058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a14dc0-3d93-4280-817b-08c348219435 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.257598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.259081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.260115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.260300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.260533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] PATH_INFO: `/v3/users/09753d93ba6b431b93ec797491cf51aa` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.268037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] RBAC: Authorizing `identity:get_user(user_id=09753d93ba6b431b93ec797491cf51aa)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.269274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.272828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 125/256] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/users/09753d93ba6b431b93ec797491cf51aa => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:59.275943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.376160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a14dc0-3d93-4280-817b-08c348219435 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.398258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6ce45-7a97-43b6-9dc8-6ce23fe6fa50 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.414967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.416059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.416443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.417306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.418489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:59.418907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.421049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.435196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 132/257] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:59.436827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.442843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.443837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.444258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.444693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.458817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:58:59.468766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:58:59.469082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:58:59.469997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:58:59.470793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 126/258] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:58:59.471446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.552531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.579976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11947e40-fb41-4f32-be85-749d73926123 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.606151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b0958-330d-4ddf-a6c3-4d6bd0855fbe None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.633741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.633741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.633741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.633741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.655944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.656922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.657226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.657537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] PATH_INFO: `/v3/users/965917eb794046faa464dfebc92e617d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.673372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] RBAC: Authorizing `identity:get_user(user_id=965917eb794046faa464dfebc92e617d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.675176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.681606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 127/259] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/users/965917eb794046faa464dfebc92e617d => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:59.682578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.719967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.722025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.754449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.794384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cd773-c7f2-4760-8395-fa733e4ee27b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.813261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:58:59.835103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.835591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.835880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.836162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.836933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:58:59.837235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:58:59.838665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:58:59.846832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 133/260] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:58:59.852633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.855333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 128/261] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:58:59.855333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:58:59.962419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ea2fe-5da9-4db1-ae9c-c666f9ee1e28 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.965646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a4cba-320b-4f1f-a216-ab13e1f91ff4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:58:59.997154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:58:59.998003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:58:59.998302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:58:59.998572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:58:59.999983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.002392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.002735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.004236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 129/262] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.009313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.009688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.010654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.011159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.011656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.053797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.088287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.090282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.109299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6bd2a-d91b-48f1-8db6-d3cdefe201f0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.153944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.154836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.155125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.155475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] PATH_INFO: `/v3/users/5ae1201438b246e0a895d8042e634e23` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.157273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] RBAC: Authorizing `identity:get_user(user_id=5ae1201438b246e0a895d8042e634e23)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.158671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.164895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 130/263] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/users/5ae1201438b246e0a895d8042e634e23 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.165750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.199738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 134/264] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:58:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:00.201346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.234193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.272759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79145b1f-2c8f-4a15-aa22-2fc9d9d34217 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.287039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.308642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.309571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.309571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.309899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.310762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] RBAC: Adding query filter params (role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=09753d93ba6b431b93ec797491cf51aa) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:00.311051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.312511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.316571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b9fcb-91b9-484e-84a1-a651aa3470da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.325750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 131/265] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/role_assignments?user.id=09753d93ba6b431b93ec797491cf51aa&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.325750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.390161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.392140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.392665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.393141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.398719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.400702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.402781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 135/266] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.412325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.439666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da73be1-bfe2-4bb3-afd0-73fb2a69c474 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.481952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.481952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.481952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.481952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.482358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.482358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.483142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 132/267] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.489427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:00.549184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.605971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:00.610233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a03a17-d9e9-42b9-ad1d-3888f38dc953 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.615940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53b28d-d769-41a7-a7ab-156927532535 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:00.657504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0feee-b02d-4e64-b972-04d1fbca1913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.658941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0feee-b02d-4e64-b972-04d1fbca1913 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.660308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0feee-b02d-4e64-b972-04d1fbca1913 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.661419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0feee-b02d-4e64-b972-04d1fbca1913 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/5ae1201438b246e0a895d8042e634e23/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.664690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0feee-b02d-4e64-b972-04d1fbca1913 None admin] RBAC: Authorizing `identity:create_grant(project_id=51d1f730296e4ce69df77b2c3a336fc3, user_id=5ae1201438b246e0a895d8042e634e23, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.666935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0feee-b02d-4e64-b972-04d1fbca1913 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.672461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7cad1-62ec-44e7-ae93-2c696ed94185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:00.673458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7cad1-62ec-44e7-ae93-2c696ed94185 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:00.673971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7cad1-62ec-44e7-ae93-2c696ed94185 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:00.674462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7cad1-62ec-44e7-ae93-2c696ed94185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:00.675506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7cad1-62ec-44e7-ae93-2c696ed94185 None admin] RBAC: Adding query filter params (role.id=01f210f8c6934006b4cb57b0fd1273bf, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=965917eb794046faa464dfebc92e617d) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:00.676061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7cad1-62ec-44e7-ae93-2c696ed94185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:00.677756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7cad1-62ec-44e7-ae93-2c696ed94185 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:00.688719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 133/268] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:00 2022] GET /identity/v3/role_assignments?user.id=965917eb794046faa464dfebc92e617d&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=01f210f8c6934006b4cb57b0fd1273bf => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:00.694732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 136/269] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:00 2022] PUT /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/5ae1201438b246e0a895d8042e634e23/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:59:01.302897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01326f-4c95-4239-b6fe-785b1d2bf9d0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.303399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01326f-4c95-4239-b6fe-785b1d2bf9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.303473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01326f-4c95-4239-b6fe-785b1d2bf9d0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.305100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 134/270] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:01.310481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de37910-42e3-4769-bb8e-7dd92222d1f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.311384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de37910-42e3-4769-bb8e-7dd92222d1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.311868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de37910-42e3-4769-bb8e-7dd92222d1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.329498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7de37910-42e3-4769-bb8e-7dd92222d1f7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:01.331040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7de37910-42e3-4769-bb8e-7dd92222d1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:01.400173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 137/271] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:01.424228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01326f-4c95-4239-b6fe-785b1d2bf9d0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.495223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2419e92-05e7-4d59-ab8e-feafc3ff8a9c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.495854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2419e92-05e7-4d59-ab8e-feafc3ff8a9c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.496335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2419e92-05e7-4d59-ab8e-feafc3ff8a9c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.498226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 138/272] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:01.507096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c681a-819e-45df-a85c-93c5ddeecbc0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.507720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c681a-819e-45df-a85c-93c5ddeecbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.508208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c681a-819e-45df-a85c-93c5ddeecbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.526143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d40c681a-819e-45df-a85c-93c5ddeecbc0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:01.527668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d40c681a-819e-45df-a85c-93c5ddeecbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:01.542068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01326f-4c95-4239-b6fe-785b1d2bf9d0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.553567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.578792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.579752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.580161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.580540 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.592247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:01.601144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:01.602228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:01.603037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:01.608135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 135/273] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:01.609055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.612287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 139/274] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:01.637939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c681a-819e-45df-a85c-93c5ddeecbc0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.653046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.734012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5b35-ca24-448e-88e1-80460f629a6f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.743493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c681a-819e-45df-a85c-93c5ddeecbc0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.769920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.770917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.771398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.771883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.772834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:01.773439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.775062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.779243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.780050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.780335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.780616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.781870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.783341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.786007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 136/275] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:01.789589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 140/276] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:01.790489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.791602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.824329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.884000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:01.905029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26b7d4-b865-4756-a8e2-1321912537dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.908913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b4fe5-5705-4cd4-854a-557c218f68e6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:01.943743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.945050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.945534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.946048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] PATH_INFO: `/v3/users/6c0a210e63cc4b488b83aaec287747fb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.947302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ca070-dc4a-47f5-abe1-be28052464b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:01.947978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca070-dc4a-47f5-abe1-be28052464b5 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:01.948263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca070-dc4a-47f5-abe1-be28052464b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:01.948547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ca070-dc4a-47f5-abe1-be28052464b5 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:01.949385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ca070-dc4a-47f5-abe1-be28052464b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.951127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746ca070-dc4a-47f5-abe1-be28052464b5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.963251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] RBAC: Authorizing `identity:get_user(user_id=6c0a210e63cc4b488b83aaec287747fb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:01.965229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:01.967750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 141/277] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/users/6c0a210e63cc4b488b83aaec287747fb => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:01.973738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:01.975221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 137/278] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:01 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:02.095341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708a59a-1b86-42e9-8d23-42281f320887 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:02.133477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f894d-73f7-46aa-b6e8-05b7cd3a231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:02.134561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f894d-73f7-46aa-b6e8-05b7cd3a231e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.135046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f894d-73f7-46aa-b6e8-05b7cd3a231e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.135521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f894d-73f7-46aa-b6e8-05b7cd3a231e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.136603 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f894d-73f7-46aa-b6e8-05b7cd3a231e None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, user.id=6c0a210e63cc4b488b83aaec287747fb, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:02.137127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f894d-73f7-46aa-b6e8-05b7cd3a231e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:02.138936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f894d-73f7-46aa-b6e8-05b7cd3a231e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:02.146347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 142/279] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 15:59:01 2022] GET /identity/v3/role_assignments?user.id=6c0a210e63cc4b488b83aaec287747fb&scope.system=all&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:02.991707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacc959-00df-46bf-8e67-60bd1656c376 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.992217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacc959-00df-46bf-8e67-60bd1656c376 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.992292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacc959-00df-46bf-8e67-60bd1656c376 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:02.993916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 138/280] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:02.999343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa779fd-3f98-43d3-b142-53113c5bd39e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:02.999810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa779fd-3f98-43d3-b142-53113c5bd39e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:02.999885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa779fd-3f98-43d3-b142-53113c5bd39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.010801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.024406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6aa779fd-3f98-43d3-b142-53113c5bd39e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:03.025608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6aa779fd-3f98-43d3-b142-53113c5bd39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:03.124185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 143/281] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:03.155456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacc959-00df-46bf-8e67-60bd1656c376 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.292693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.297193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e87707-9513-442d-b714-6fcc9b1ddcbe None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.297511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e87707-9513-442d-b714-6fcc9b1ddcbe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.297886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e87707-9513-442d-b714-6fcc9b1ddcbe None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.305738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 144/282] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:03.307026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7749-6f4c-413a-9879-f232c33f0df7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.307317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7749-6f4c-413a-9879-f232c33f0df7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.307602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7749-6f4c-413a-9879-f232c33f0df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.333298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a87f7749-6f4c-413a-9879-f232c33f0df7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:03.333298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a87f7749-6f4c-413a-9879-f232c33f0df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:03.333948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacc959-00df-46bf-8e67-60bd1656c376 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.411042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.411868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.412157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.412439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.417158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 145/283] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:03.439701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:03.441259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f7749-6f4c-413a-9879-f232c33f0df7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:03.458343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:03.459294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 139/284] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:03.460345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.558175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.624344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f7749-6f4c-413a-9879-f232c33f0df7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.655408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0436b757-5b38-4c1a-9ced-b8192dc5d2c5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.691136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.691944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.692229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.692508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.710959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:03.719980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:03.720864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:03.721672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 146/285] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:03.729631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.730335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.730625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.730899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.731649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:03.731959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.733341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.736955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.752149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 140/286] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:03.759211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.772525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.872017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abee065-7d04-42be-bb61-4e2cd1355b67 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.927769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eb6be-6ed6-4790-a778-476ab778bd3a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:03.952839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:03.965629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.966639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.967134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.967587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.968480 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:03.968934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.970690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.974437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:03.975341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:03.975633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:03.975918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] PATH_INFO: `/v3/users/5ae1201438b246e0a895d8042e634e23` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:03.977302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] RBAC: Authorizing `identity:get_user(user_id=5ae1201438b246e0a895d8042e634e23)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:03.978742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:03.980710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 141/287] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/users/5ae1201438b246e0a895d8042e634e23 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:03.986054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 147/288] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:03.987186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:03.991905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.098235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebddb8-be88-4495-a6f3-a45e2dec7d78 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.147340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.174034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dece1de-6d39-42fb-9c6d-5ec0bb3b3574 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.190458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.191361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.191654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.191938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] PATH_INFO: `/v3/users/965917eb794046faa464dfebc92e617d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.198479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] RBAC: Authorizing `identity:get_user(user_id=965917eb794046faa464dfebc92e617d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.205812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.205812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 148/289] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/users/965917eb794046faa464dfebc92e617d => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:04.209573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.223601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.225403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.225922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.226404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.303332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.305180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.355443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.387875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.414350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 142/290] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:59:04.420147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.467965 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9731527-566c-45cb-ab91-5faab642cf2c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.527904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf4495-bffb-48a8-bb0c-26237903a79a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.565046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.566316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.566438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.566958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.567311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.567604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.567825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.567921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.569124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.569615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.570612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.571497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.572639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 149/291] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:04 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:04.582181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 143/292] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:04 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:04.583083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.585611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.636600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.702488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f6bd7-8a0d-435b-941b-e6fe6902ce43 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.739821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e73e9-7924-466d-b5ca-c78fc17cc288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.741055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e73e9-7924-466d-b5ca-c78fc17cc288 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.741339 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e73e9-7924-466d-b5ca-c78fc17cc288 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.741613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e73e9-7924-466d-b5ca-c78fc17cc288 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/965917eb794046faa464dfebc92e617d/roles/01f210f8c6934006b4cb57b0fd1273bf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.743600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e73e9-7924-466d-b5ca-c78fc17cc288 None admin] RBAC: Authorizing `identity:create_grant(project_id=51d1f730296e4ce69df77b2c3a336fc3, user_id=965917eb794046faa464dfebc92e617d, role_id=01f210f8c6934006b4cb57b0fd1273bf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.745144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e73e9-7924-466d-b5ca-c78fc17cc288 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.761373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 144/293] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:04 2022] PUT /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/users/965917eb794046faa464dfebc92e617d/roles/01f210f8c6934006b4cb57b0fd1273bf => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:59:04.762541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87da4-4b71-43aa-a5f4-03baab52df46 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.762835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87da4-4b71-43aa-a5f4-03baab52df46 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.763116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b87da4-4b71-43aa-a5f4-03baab52df46 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.764754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 145/294] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:04.765917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd5aca-e56c-450a-893c-5ef524f24975 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.766186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd5aca-e56c-450a-893c-5ef524f24975 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.766462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd5aca-e56c-450a-893c-5ef524f24975 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.772278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 146/295] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:04.779373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c73739-e239-4de7-bbd0-0d6decfed05e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.779742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c73739-e239-4de7-bbd0-0d6decfed05e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.779939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c73739-e239-4de7-bbd0-0d6decfed05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.803040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600f755-04f8-434b-b55d-c5683fe33324 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:04.807390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-84c73739-e239-4de7-bbd0-0d6decfed05e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:04.808633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-84c73739-e239-4de7-bbd0-0d6decfed05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:04.839741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:04.852950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e205a8-6d9f-4358-96fc-14da8c368051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:04.853638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e205a8-6d9f-4358-96fc-14da8c368051 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.853964 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e205a8-6d9f-4358-96fc-14da8c368051 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.854249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e205a8-6d9f-4358-96fc-14da8c368051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.855109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e205a8-6d9f-4358-96fc-14da8c368051 None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=5ae1201438b246e0a895d8042e634e23) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:04.855448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e205a8-6d9f-4358-96fc-14da8c368051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:04.856977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e205a8-6d9f-4358-96fc-14da8c368051 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:04.870398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 150/296] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:04 2022] GET /identity/v3/role_assignments?user.id=5ae1201438b246e0a895d8042e634e23&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:04.870398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba8a26-9d24-49b8-8a7f-a70478eb3ceb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.870398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba8a26-9d24-49b8-8a7f-a70478eb3ceb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.870398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba8a26-9d24-49b8-8a7f-a70478eb3ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.890233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-34ba8a26-9d24-49b8-8a7f-a70478eb3ceb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:04.891681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34ba8a26-9d24-49b8-8a7f-a70478eb3ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:04.903712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 147/297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:04.905244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8712-241d-4f13-b8fd-2cd29ea82f30 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.905545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8712-241d-4f13-b8fd-2cd29ea82f30 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.905850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa8712-241d-4f13-b8fd-2cd29ea82f30 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.907388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 148/298] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:04.923898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd9f1-ca83-4616-b329-051f626792ee None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:04.924210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd9f1-ca83-4616-b329-051f626792ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:04.924467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd9f1-ca83-4616-b329-051f626792ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:04.950643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d31cd9f1-ca83-4616-b329-051f626792ee None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:04.952294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d31cd9f1-ca83-4616-b329-051f626792ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:04.988632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 151/299] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:04.989847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba8a26-9d24-49b8-8a7f-a70478eb3ceb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:04.998191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.055771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 149/300] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:05.056740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cd9f1-ca83-4616-b329-051f626792ee None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.131568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba8a26-9d24-49b8-8a7f-a70478eb3ceb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.152547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.185869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.186694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.186983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.187264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.201776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:05.202976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cd9f1-ca83-4616-b329-051f626792ee None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:05.209900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:05.211087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 152/301] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:05.213119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.254554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.255614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.256075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.256531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.258523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.260849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.263920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 150/302] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:05.265191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.313628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.356407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81945965-57ff-4420-9a42-a37c5da4d5f7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.392908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.393713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.394029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.394312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.402311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304cdad-3c03-425e-bdb6-93b6d6405536 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.406064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:05.414721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:05.415828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 153/303] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:05.416803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.448437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.449967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.450497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.450918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.452461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:05.453292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.456016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.460588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.472248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 151/304] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:05.472248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.539680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccfa24-ba19-4842-90b9-d1e917494c1b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.579128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.580437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.581468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.582958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.583346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] PATH_INFO: `/v3/groups` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.586693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.588761 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.590369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93f904-997f-4eb7-bd90-1c9c8efb36e5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.608395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 154/305] 10.222.0.88 () {60 vars in 1134 bytes} [Thu Jun 16 15:59:05 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:05.609225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.654609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.655311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.655600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.655877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.656776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:05.657090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.658941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.678194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 152/306] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:05.679698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.745615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.754761 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9620e2-24c6-460a-9ef7-d4c7c94d185f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.808468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.810565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.811079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.811535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] PATH_INFO: `/v3/users/6c0a210e63cc4b488b83aaec287747fb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.814298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] RBAC: Authorizing `identity:get_user(user_id=6c0a210e63cc4b488b83aaec287747fb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.816809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.819608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 155/307] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/users/6c0a210e63cc4b488b83aaec287747fb => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:05.825396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.847615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd1f89-14d2-4229-aac4-829945847573 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:05.892732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:05.893863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:05.894227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:05.894587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] PATH_INFO: `/v3/users/2d79403a71794d72aa957da16cc7c62b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:05.914618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] RBAC: Authorizing `identity:get_user(user_id=2d79403a71794d72aa957da16cc7c62b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:05.916282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:05.918582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 153/308] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/users/2d79403a71794d72aa957da16cc7c62b => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:05.924456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:05.949053 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:05.969657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8243f2-cda3-407f-806f-f30c54e3ce12 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.018771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c53b6-4d93-4f1b-9e27-219940c8bbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.020026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c53b6-4d93-4f1b-9e27-219940c8bbc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.020720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c53b6-4d93-4f1b-9e27-219940c8bbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.021159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c53b6-4d93-4f1b-9e27-219940c8bbc6 None admin] PATH_INFO: `/v3/system/users/6c0a210e63cc4b488b83aaec287747fb/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.024426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c53b6-4d93-4f1b-9e27-219940c8bbc6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6c0a210e63cc4b488b83aaec287747fb, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.025874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c53b6-4d93-4f1b-9e27-219940c8bbc6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.049753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 156/309] 10.222.0.88 () {58 vars in 1334 bytes} [Thu Jun 16 15:59:05 2022] PUT /identity/v3/system/users/6c0a210e63cc4b488b83aaec287747fb/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:59:06.074332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b74be3-d6a8-4250-898f-0e5d1abb74bc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:06.115872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f52c1-8825-4226-8519-1941b17adb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:06.116557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f52c1-8825-4226-8519-1941b17adb55 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:06.116842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f52c1-8825-4226-8519-1941b17adb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:06.117118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f52c1-8825-4226-8519-1941b17adb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:06.118073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f52c1-8825-4226-8519-1941b17adb55 None admin] RBAC: Adding query filter params (role.id=01f210f8c6934006b4cb57b0fd1273bf, user.id=2d79403a71794d72aa957da16cc7c62b, scope.system=all) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:06.118396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f52c1-8825-4226-8519-1941b17adb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:06.120002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413f52c1-8825-4226-8519-1941b17adb55 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:06.127502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 154/310] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 15:59:05 2022] GET /identity/v3/role_assignments?user.id=2d79403a71794d72aa957da16cc7c62b&scope.system=all&role.id=01f210f8c6934006b4cb57b0fd1273bf => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:07.049809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1e902-7162-4d87-a0f2-48aef454e3ec None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.050325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1e902-7162-4d87-a0f2-48aef454e3ec None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.050401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1e902-7162-4d87-a0f2-48aef454e3ec None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.051981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 157/311] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:07.057422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d74a61-dbd0-425e-ab26-4c47d2b888fd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.057856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d74a61-dbd0-425e-ab26-4c47d2b888fd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.058084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d74a61-dbd0-425e-ab26-4c47d2b888fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.075929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d6d74a61-dbd0-425e-ab26-4c47d2b888fd None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:07.077146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d6d74a61-dbd0-425e-ab26-4c47d2b888fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:07.164696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 155/312] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:07.189348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1e902-7162-4d87-a0f2-48aef454e3ec None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.220748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.315944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1e902-7162-4d87-a0f2-48aef454e3ec None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.354265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.355096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.355385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.355664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.369876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:07.377751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:07.378709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:07.379518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 158/313] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:07.385279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d74a61-dbd0-425e-ab26-4c47d2b888fd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.403184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.510896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d74a61-dbd0-425e-ab26-4c47d2b888fd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.554329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.555012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.555296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.555571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.556322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:07.556630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.558033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.570441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 156/314] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:07.576739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.687962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.707865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cbc21-32d0-4e99-b138-bc9ff7956777 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.748086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36401bc5-bdbe-4093-a108-45423adb1f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.748979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36401bc5-bdbe-4093-a108-45423adb1f5b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.749267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36401bc5-bdbe-4093-a108-45423adb1f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.749547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36401bc5-bdbe-4093-a108-45423adb1f5b None admin] PATH_INFO: `/v3/users/965917eb794046faa464dfebc92e617d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.751036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36401bc5-bdbe-4093-a108-45423adb1f5b None admin] RBAC: Authorizing `identity:get_user(user_id=965917eb794046faa464dfebc92e617d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:07.752643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36401bc5-bdbe-4093-a108-45423adb1f5b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:07.754734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 159/315] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/users/965917eb794046faa464dfebc92e617d => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:07.760027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:07.847303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:07.876238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07ffd-e231-473a-a1ac-275dc84be7e3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:07.916755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:07.917443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.917763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.918034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.990112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2ad9-c4ff-4b47-ad35-4f2721db84a9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.990432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2ad9-c4ff-4b47-ad35-4f2721db84a9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.990666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a2ad9-c4ff-4b47-ad35-4f2721db84a9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:07.992385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 160/316] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:07.998675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43e556-1b55-4b98-bebe-14e74279fe53 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:07.999302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43e556-1b55-4b98-bebe-14e74279fe53 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:07.999903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43e556-1b55-4b98-bebe-14e74279fe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.004444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.009420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.028523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cb43e556-1b55-4b98-bebe-14e74279fe53 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:08.030727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb43e556-1b55-4b98-bebe-14e74279fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:08.115860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.127497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 161/317] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:08.153998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 157/318] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:08.156527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43e556-1b55-4b98-bebe-14e74279fe53 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.161017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.275443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.290826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece01cb-bd88-428d-8bad-337b5000268e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.299746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43e556-1b55-4b98-bebe-14e74279fe53 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.338561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.339439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.339810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.340253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.341756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.343638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.346651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 158/319] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:08.347582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.348586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.349031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.349452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.351143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.352793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.354727 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.356845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 162/320] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:08.362620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.419803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.500455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2df74-b356-4db2-9066-957183a78d7b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.503026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17574f3e-32bd-4286-a38c-c61631d44248 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.543999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332a3ee-d66d-438b-807f-f28e9e894c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.544705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332a3ee-d66d-438b-807f-f28e9e894c1d None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.544992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332a3ee-d66d-438b-807f-f28e9e894c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.545282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332a3ee-d66d-438b-807f-f28e9e894c1d None admin] PATH_INFO: `/v3/groups` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.546019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332a3ee-d66d-438b-807f-f28e9e894c1d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.546223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff688e90-3e93-4081-be5f-09500c7a3bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.546848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff688e90-3e93-4081-be5f-09500c7a3bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.547134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff688e90-3e93-4081-be5f-09500c7a3bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.547412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff688e90-3e93-4081-be5f-09500c7a3bb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.547503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332a3ee-d66d-438b-807f-f28e9e894c1d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.548805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff688e90-3e93-4081-be5f-09500c7a3bb7 None admin] RBAC: Adding query filter params (role.id=01f210f8c6934006b4cb57b0fd1273bf, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3, user.id=965917eb794046faa464dfebc92e617d) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:08.549118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff688e90-3e93-4081-be5f-09500c7a3bb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:08.550641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff688e90-3e93-4081-be5f-09500c7a3bb7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:08.557979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 159/321] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/role_assignments?user.id=965917eb794046faa464dfebc92e617d&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=01f210f8c6934006b4cb57b0fd1273bf => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:08.565071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 163/322] 10.222.0.88 () {60 vars in 1134 bytes} [Thu Jun 16 15:59:08 2022] POST /identity/v3/groups => generated 226 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:08.568554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5e1a1-df98-4b76-a684-20f280240883 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.568848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5e1a1-df98-4b76-a684-20f280240883 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.569118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5e1a1-df98-4b76-a684-20f280240883 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.577769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 160/323] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:08.580389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14c6a-4d3c-43da-a096-e6b0a9176dad None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.580693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14c6a-4d3c-43da-a096-e6b0a9176dad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.580970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a14c6a-4d3c-43da-a096-e6b0a9176dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.593946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e05785-8f45-4521-ad1a-aac407e942ed None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.594241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e05785-8f45-4521-ad1a-aac407e942ed None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.594521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e05785-8f45-4521-ad1a-aac407e942ed None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.596047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 161/324] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:08.601324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-60a14c6a-4d3c-43da-a096-e6b0a9176dad None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:08.602152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e58cb-899b-47c8-8b71-917486ec213f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.602447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e58cb-899b-47c8-8b71-917486ec213f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.602734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e58cb-899b-47c8-8b71-917486ec213f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.603603 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-60a14c6a-4d3c-43da-a096-e6b0a9176dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:08.616191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.622545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-965e58cb-899b-47c8-8b71-917486ec213f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:08.623855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-965e58cb-899b-47c8-8b71-917486ec213f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:08.706535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 164/325] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:08.718806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 162/326] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:08.730030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a14c6a-4d3c-43da-a096-e6b0a9176dad None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.745229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e58cb-899b-47c8-8b71-917486ec213f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.789666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.880659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a14c6a-4d3c-43da-a096-e6b0a9176dad None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.890499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e58cb-899b-47c8-8b71-917486ec213f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:08.925500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.927286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.927801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.928259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.941979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:08.942513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:08.943322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:08.943601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:08.943874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:08.950151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:08.951094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:08.951899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:08.952497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 165/327] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:08.958370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:08.966975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:08.969392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:08.971312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:08.973200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:08.974119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 163/328] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:08.979681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.121963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.133770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097cc16-56b1-455b-803d-7ee6f8efe758 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.133957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a487-bc8c-446a-abdc-b8e1feccde99 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.180449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.181206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.181496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.181812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.182599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:09.183015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.184544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.194265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 164/329] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:09.200742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.205154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.206397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.206892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.207384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.208820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:09.209307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.212290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.230623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 166/330] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:09.239386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.316137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4232d-2b9f-4cd1-b6d4-f4b25add44f6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.356622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.357888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.358181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.358462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] PATH_INFO: `/v3/users/2d79403a71794d72aa957da16cc7c62b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.360122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] RBAC: Authorizing `identity:get_user(user_id=2d79403a71794d72aa957da16cc7c62b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.362239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.365490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 165/331] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:09 2022] GET /identity/v3/users/2d79403a71794d72aa957da16cc7c62b => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:09.370629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.375099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.417280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.435551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a36d49-a8df-43be-9fe5-35519b864c5c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.479367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.480627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.481115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.481668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] PATH_INFO: `/v3/users/6c0a210e63cc4b488b83aaec287747fb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.483240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] RBAC: Authorizing `identity:get_user(user_id=6c0a210e63cc4b488b83aaec287747fb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.485070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.487942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 167/332] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:09 2022] GET /identity/v3/users/6c0a210e63cc4b488b83aaec287747fb => generated 293 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:09.493592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:09.512767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4d022-f19d-4bb0-ae8c-8777ed6b4d27 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.554715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0337e4-9744-4ba8-8a49-e08bb3d6b764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.555350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0337e4-9744-4ba8-8a49-e08bb3d6b764 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.555637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0337e4-9744-4ba8-8a49-e08bb3d6b764 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.555921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0337e4-9744-4ba8-8a49-e08bb3d6b764 None admin] PATH_INFO: `/v3/system/users/2d79403a71794d72aa957da16cc7c62b/roles/01f210f8c6934006b4cb57b0fd1273bf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.557465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0337e4-9744-4ba8-8a49-e08bb3d6b764 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d79403a71794d72aa957da16cc7c62b, role_id=01f210f8c6934006b4cb57b0fd1273bf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.559343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0337e4-9744-4ba8-8a49-e08bb3d6b764 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.571184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 166/333] 10.222.0.88 () {58 vars in 1334 bytes} [Thu Jun 16 15:59:09 2022] PUT /identity/v3/system/users/2d79403a71794d72aa957da16cc7c62b/roles/01f210f8c6934006b4cb57b0fd1273bf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:09.644408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbaedf-0113-4e52-86a5-dc334ca18743 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:09.687022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a2975-b556-468d-b38b-b90b7155f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:09.688035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a2975-b556-468d-b38b-b90b7155f626 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:09.688486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a2975-b556-468d-b38b-b90b7155f626 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:09.689456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a2975-b556-468d-b38b-b90b7155f626 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:09.690721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a2975-b556-468d-b38b-b90b7155f626 None admin] RBAC: Adding query filter params (role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, user.id=6c0a210e63cc4b488b83aaec287747fb, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:09.691212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a2975-b556-468d-b38b-b90b7155f626 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:09.692908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a2975-b556-468d-b38b-b90b7155f626 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:09.695391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:09.708286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 168/334] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 15:59:09 2022] GET /identity/v3/role_assignments?user.id=6c0a210e63cc4b488b83aaec287747fb&scope.system=all&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:11.491209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8da55-704d-4d8c-8aa4-f68ef8ae7fb4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.492017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8da55-704d-4d8c-8aa4-f68ef8ae7fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.492017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8da55-704d-4d8c-8aa4-f68ef8ae7fb4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.493454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 167/335] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:11.499285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8b8d7-31ea-4dae-8b87-e59ad9686a0b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.500066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8b8d7-31ea-4dae-8b87-e59ad9686a0b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.500066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8b8d7-31ea-4dae-8b87-e59ad9686a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.523515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-72d8b8d7-31ea-4dae-8b87-e59ad9686a0b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:11.525559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6e061-e0c0-462c-8469-8a11d9ae82f5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.525875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6e061-e0c0-462c-8469-8a11d9ae82f5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.526160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6e061-e0c0-462c-8469-8a11d9ae82f5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.527682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 168/336] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:11.528443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72d8b8d7-31ea-4dae-8b87-e59ad9686a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:11.534983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281524-880c-4ef5-b13f-a42f3b2f43ba None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.535279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281524-880c-4ef5-b13f-a42f3b2f43ba None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.535545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac281524-880c-4ef5-b13f-a42f3b2f43ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.567097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ac281524-880c-4ef5-b13f-a42f3b2f43ba None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:11.568303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ac281524-880c-4ef5-b13f-a42f3b2f43ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:11.702293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 169/337] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:11.707347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 169/338] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:11.746895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281524-880c-4ef5-b13f-a42f3b2f43ba None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:11.759201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8b8d7-31ea-4dae-8b87-e59ad9686a0b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:11.776534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:11.881955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac281524-880c-4ef5-b13f-a42f3b2f43ba None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:11.920703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:11.921972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:11.922504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:11.923055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:11.934932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:11.944046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:11.945012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:11.946357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:11.953646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 170/339] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:11.955702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.023736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8b8d7-31ea-4dae-8b87-e59ad9686a0b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.051177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.072978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29d3d4-1139-42af-b9ec-251e78c2b0fe None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.075566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.112761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.114925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.115318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.115694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.116562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:12.116978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.118695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.133971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 171/340] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:12.136552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.158990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.159841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.160130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.160411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.214862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:12.246000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9d407-0764-4871-8686-4b6da6c7613b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:12.247506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:12.248377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:12.249168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:12.250251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 170/341] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 491 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:12.256754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.282723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.283374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.283665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.283945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.311175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.359608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.361632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.497860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 172/342] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:59:12.504468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.563568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.583682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9101b-845c-4036-a64b-77f747f35873 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.614526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de018fea-f18b-4def-9e7f-cdac5b169e9b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.650773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38f718-1ee5-43ec-a2ee-f65825e1ad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.651652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f718-1ee5-43ec-a2ee-f65825e1ad86 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.651940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f718-1ee5-43ec-a2ee-f65825e1ad86 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.652223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38f718-1ee5-43ec-a2ee-f65825e1ad86 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.653439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38f718-1ee5-43ec-a2ee-f65825e1ad86 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.654489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbbdfd9-a63c-4f30-857e-c06d10e98d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.654925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38f718-1ee5-43ec-a2ee-f65825e1ad86 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.655188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbdfd9-a63c-4f30-857e-c06d10e98d44 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.655481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbdfd9-a63c-4f30-857e-c06d10e98d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.655757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbbdfd9-a63c-4f30-857e-c06d10e98d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.656521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbdfd9-a63c-4f30-857e-c06d10e98d44 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:12.656632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 173/343] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:12.657046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbdfd9-a63c-4f30-857e-c06d10e98d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.658854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbbdfd9-a63c-4f30-857e-c06d10e98d44 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.659473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f444-67a3-43da-8683-0147243fbc21 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.659762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f444-67a3-43da-8683-0147243fbc21 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.660040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575f444-67a3-43da-8683-0147243fbc21 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.661307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 174/344] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:12.666098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f444-67a3-43da-8683-0147243fbc21 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.668631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 171/345] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:12.674574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae4f72-f92a-470a-9247-ad33529171f9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.674886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae4f72-f92a-470a-9247-ad33529171f9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.675173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae4f72-f92a-470a-9247-ad33529171f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.698734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4ae4f72-f92a-470a-9247-ad33529171f9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:12.699927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4ae4f72-f92a-470a-9247-ad33529171f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:12.770283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.777604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575f444-67a3-43da-8683-0147243fbc21 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.794106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:12.820070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 172/346] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:12.820852 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae4f72-f92a-470a-9247-ad33529171f9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.821413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.822648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.822944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.823232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:12.838341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:12.839839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:12.842109 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 175/347] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:12.848443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:12.961833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf8827-67c5-46dd-8227-3dfa82a208b2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.976060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae4f72-f92a-470a-9247-ad33529171f9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:12.998521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:12.999257 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:12.999544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:12.999831 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.000738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=caaea6a33f9840079019e777b132555a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:13.001063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.003099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.013913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 176/348] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:13.015033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.042491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.043177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.043463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.043740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.070216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.122391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.146708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41870871-384a-4dc8-811e-bf0a7333db47 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.165309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.166872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.184493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.185324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.185619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.185937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.198866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:13.214265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:13.215267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:13.216085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 177/349] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:13.223220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.328954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 173/350] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:13.335825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.340252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84366ee2-2fdd-4ba6-8c36-657682713941 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.366133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.380448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.381148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.381450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.381762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.383211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:13.383528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.385092 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.398925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 178/351] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:13.400316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.466604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.504743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2ba80-9d73-4403-b1de-569066ff62fd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.526169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20066da4-fd7b-4d10-a7ee-fd70c30f4869 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.563616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9b4a5-8ca0-4f21-88a4-fa46e7d9679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.565032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b4a5-8ca0-4f21-88a4-fa46e7d9679e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.565316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b4a5-8ca0-4f21-88a4-fa46e7d9679e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.565594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b4a5-8ca0-4f21-88a4-fa46e7d9679e None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.566539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.566874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b4a5-8ca0-4f21-88a4-fa46e7d9679e None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.567214 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.567504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.567787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.568338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b4a5-8ca0-4f21-88a4-fa46e7d9679e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.570443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 174/352] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:13.571695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f58ad-4ddb-4618-90ec-0ce9e3068694 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.571984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f58ad-4ddb-4618-90ec-0ce9e3068694 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.572264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f58ad-4ddb-4618-90ec-0ce9e3068694 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.573776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 175/353] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:13.580305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ed548-a4e4-4325-a9c0-8d6404c6a2f4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.580731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ed548-a4e4-4325-a9c0-8d6404c6a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.581203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ed548-a4e4-4325-a9c0-8d6404c6a2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.600138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.611823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cd0ed548-a4e4-4325-a9c0-8d6404c6a2f4 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:13.613318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd0ed548-a4e4-4325-a9c0-8d6404c6a2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:13.647210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.648711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.726286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 176/354] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:13.728102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ed548-a4e4-4325-a9c0-8d6404c6a2f4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.793240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 179/355] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:13.794230 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.826765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.869292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ed548-a4e4-4325-a9c0-8d6404c6a2f4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.886544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:13.919522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.920715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.921172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.921491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:13.926268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:13.927834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:13.930218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 177/356] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:13.931123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:13.947676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dae741-65e0-4241-96bb-55244ac6ac98 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:13.991431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:13.992848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:13.993255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:13.993662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.021376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:14.034986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:14.035953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:14.036808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:14.040345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 180/357] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:14.046391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.149748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7359c-0f96-4a6f-9ea0-2d26cb8b90ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.165759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec164cb8-059a-41d3-a417-b9196f04e8a1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.184407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.196003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.196900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.197187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.197473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.200872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.202946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.205390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 178/358] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:13 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:14.206408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.209685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937ec4a-716f-48ac-b879-61832d2ad310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.210487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937ec4a-716f-48ac-b879-61832d2ad310 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.210781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937ec4a-716f-48ac-b879-61832d2ad310 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.211071 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937ec4a-716f-48ac-b879-61832d2ad310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.212190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937ec4a-716f-48ac-b879-61832d2ad310 None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=caaea6a33f9840079019e777b132555a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:14.212510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937ec4a-716f-48ac-b879-61832d2ad310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.214022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937ec4a-716f-48ac-b879-61832d2ad310 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.218521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.221614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 181/359] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:14.223202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502ff7c-059a-4301-9f75-aa07440d660e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.223499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502ff7c-059a-4301-9f75-aa07440d660e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.223783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502ff7c-059a-4301-9f75-aa07440d660e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.225177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 182/360] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:14.227335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502ff7c-059a-4301-9f75-aa07440d660e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.358574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502ff7c-059a-4301-9f75-aa07440d660e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.377595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823ed04-1b6a-4b6a-8399-98ec0f1a9ad5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.402129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18166539-0a24-4ca7-a199-3b0fa5f198ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.402977 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18166539-0a24-4ca7-a199-3b0fa5f198ce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.403267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18166539-0a24-4ca7-a199-3b0fa5f198ce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.403549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18166539-0a24-4ca7-a199-3b0fa5f198ce None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.404935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18166539-0a24-4ca7-a199-3b0fa5f198ce None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.406583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18166539-0a24-4ca7-a199-3b0fa5f198ce None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.408960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 183/361] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:14.415342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a9f5-d2b1-417a-a46e-5e94dd24bca0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.415675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a9f5-d2b1-417a-a46e-5e94dd24bca0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.415944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a9f5-d2b1-417a-a46e-5e94dd24bca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.435287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b0c2a9f5-d2b1-417a-a46e-5e94dd24bca0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:14.436524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b0c2a9f5-d2b1-417a-a46e-5e94dd24bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:14.443129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.443827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.444115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.444394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.445160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:14.445472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.446929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.466977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 179/362] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:14.468045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.489985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.514903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 184/363] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:14.515973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2a9f5-d2b1-417a-a46e-5e94dd24bca0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.562377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.603314 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a56e17-30d2-430f-870c-b5982a643548 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.627963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2a9f5-d2b1-417a-a46e-5e94dd24bca0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.643620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.644876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.645317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.645918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.647031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:14.647501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.649108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.656103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 180/364] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:14.657084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.672456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e03ede-b2bd-4969-a933-1b92df4908c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.673132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e03ede-b2bd-4969-a933-1b92df4908c9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.673420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e03ede-b2bd-4969-a933-1b92df4908c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.673768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e03ede-b2bd-4969-a933-1b92df4908c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.743659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:14.763491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e03ede-b2bd-4969-a933-1b92df4908c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:14.765009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e03ede-b2bd-4969-a933-1b92df4908c9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:14.884560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 185/365] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:14.885863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c344c-c5ef-463c-aa47-ee7931daf896 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.886156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c344c-c5ef-463c-aa47-ee7931daf896 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.886439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c344c-c5ef-463c-aa47-ee7931daf896 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.889780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 186/366] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:14.894840 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c344c-c5ef-463c-aa47-ee7931daf896 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:14.925289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c8b99-8b92-4890-93bd-3017760794f2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:14.987284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac6273-7101-489c-b2ce-b695de1546bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:14.988333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac6273-7101-489c-b2ce-b695de1546bb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:14.988657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac6273-7101-489c-b2ce-b695de1546bb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:14.988981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac6273-7101-489c-b2ce-b695de1546bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:14.991307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.004939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac6273-7101-489c-b2ce-b695de1546bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:15.013941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c2ac6273-7101-489c-b2ce-b695de1546bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:15.017656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:15.018578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:15.019385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 181/367] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:15.022201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd447a31-d9ac-4c47-8b68-9b0f95c90f8b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.022201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd447a31-d9ac-4c47-8b68-9b0f95c90f8b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.022201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd447a31-d9ac-4c47-8b68-9b0f95c90f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.043473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bd447a31-d9ac-4c47-8b68-9b0f95c90f8b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:15.044844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bd447a31-d9ac-4c47-8b68-9b0f95c90f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:15.058617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c344c-c5ef-463c-aa47-ee7931daf896 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.130756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.131642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.131928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.132207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.136232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.138102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.140334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 187/368] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:14 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:15.141130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.162433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 182/369] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:15.163306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd447a31-d9ac-4c47-8b68-9b0f95c90f8b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.291267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228fb55-f15c-4988-ae0c-5577cc444080 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.292406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd447a31-d9ac-4c47-8b68-9b0f95c90f8b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.317371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.329763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.331550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.332009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.332438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.333240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.333866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.334186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.334475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.334758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.335514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:15.335826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.336275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.337454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.338507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 183/370] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:15.339516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.347598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 188/371] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:15.352954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.384401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.453853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050c453-7a9b-4f0d-8117-371c9836610e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.491635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.492454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.492750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.493030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.517273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:15.525481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:15.526431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:15.527245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:15.535841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 184/372] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:15.537322 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.545166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413dfec-f568-468d-9da1-eb9780be1417 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.606366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.614627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.615709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.616043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.616448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] PATH_INFO: `/v3/users/2d79403a71794d72aa957da16cc7c62b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.618278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] RBAC: Authorizing `identity:get_user(user_id=2d79403a71794d72aa957da16cc7c62b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.620056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.622102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 189/373] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/users/2d79403a71794d72aa957da16cc7c62b => generated 293 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:15.622894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.671541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da269d2-7748-4e52-b502-11a6d3600723 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.716000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.716680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.716970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.717254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.726216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:15.726556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.726962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.728042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.734892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 185/374] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:15.735728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.752502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173753b-9e80-4050-8ba8-fbfe97fe5c69 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.794227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.795047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.795450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.795698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.796492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:15.796930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.798577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.808964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 190/375] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:15.815009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:15.874218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd75eb-7553-439c-a25c-522a6869501f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:15.898705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:15.914582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:15.915450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:15.915752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:15.916043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:15.917341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] RBAC: Adding query filter params (role.id=01f210f8c6934006b4cb57b0fd1273bf, user.id=2d79403a71794d72aa957da16cc7c62b, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:15.917681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:15.919400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:15.935458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 186/376] 10.222.0.88 () {56 vars in 1308 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/role_assignments?user.id=2d79403a71794d72aa957da16cc7c62b&scope.system=all&role.id=01f210f8c6934006b4cb57b0fd1273bf => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:16.023318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fad40-49eb-4e6e-9e6a-92ad6d48f691 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:16.081591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:16.082416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.082707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.083025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.098406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:16.183172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:16.184744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:16.299504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 191/377] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:59:16.304771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:16.413500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa1ce5-4272-43cd-862f-0767ed5e43cd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:16.449411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:16.450311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.450622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.450941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] PATH_INFO: `/v3/projects/a913cabdc8c6449f971e776cd4ba8813` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.452162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] RBAC: Authorizing `identity:get_project(project_id=a913cabdc8c6449f971e776cd4ba8813)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:16.453623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:16.455757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 187/378] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:16 2022] GET /identity/v3/projects/a913cabdc8c6449f971e776cd4ba8813 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:16.460963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:16.507241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:16.584892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea83458-74df-45d1-a622-edf73eb0301c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:16.631669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843ee75-9ef8-4c0d-9ad7-53c5aafbc022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:16.632565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843ee75-9ef8-4c0d-9ad7-53c5aafbc022 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.632849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843ee75-9ef8-4c0d-9ad7-53c5aafbc022 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.633127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843ee75-9ef8-4c0d-9ad7-53c5aafbc022 None admin] PATH_INFO: `/v3/groups/961912e2a2c644b9b5226d1397c8f5bb` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.641611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843ee75-9ef8-4c0d-9ad7-53c5aafbc022 None admin] RBAC: Authorizing `identity:get_group(group_id=961912e2a2c644b9b5226d1397c8f5bb)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:16.643738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843ee75-9ef8-4c0d-9ad7-53c5aafbc022 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:16.645858 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 192/379] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:16 2022] GET /identity/v3/groups/961912e2a2c644b9b5226d1397c8f5bb => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:16.651959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:16.694170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09fcbd-cbd8-441a-8368-94832ac3b523 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.694170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09fcbd-cbd8-441a-8368-94832ac3b523 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.694170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09fcbd-cbd8-441a-8368-94832ac3b523 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.695239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 193/380] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:16.702136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50684f-f6e1-4249-9d7e-fd612d1fc755 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.702560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50684f-f6e1-4249-9d7e-fd612d1fc755 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.702886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50684f-f6e1-4249-9d7e-fd612d1fc755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.703749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:16.725390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4e50684f-f6e1-4249-9d7e-fd612d1fc755 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:16.726651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4e50684f-f6e1-4249-9d7e-fd612d1fc755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:16.778308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7567640-a9cc-4732-ac30-11ddeb13f6ca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:16.806541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 194/381] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:16.835915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:16.836820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:16.837503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:16.837979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:16.838966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] RBAC: Adding query filter params (group.id=961912e2a2c644b9b5226d1397c8f5bb, role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=a913cabdc8c6449f971e776cd4ba8813) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:16.839424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:16.841014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:16.843548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50684f-f6e1-4249-9d7e-fd612d1fc755 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:16.850152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 188/382] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:16 2022] GET /identity/v3/role_assignments?group.id=961912e2a2c644b9b5226d1397c8f5bb&scope.project.id=a913cabdc8c6449f971e776cd4ba8813&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:16.908069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:16.993943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50684f-f6e1-4249-9d7e-fd612d1fc755 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.048331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.049207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.049458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.049744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.066691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:17.075481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:17.076417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:17.077229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:17.077885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 195/383] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:17.083970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.253708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11aa73-23d8-4db8-97fa-b7ab0f08001b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.261369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.299938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.300696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.300985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.301270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.302053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:17.302366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.303762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.312976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 189/384] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:17.318826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.453223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.467046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229788fb-f7cd-4824-817c-9c47214463f9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.509171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.510472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.510953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.511457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.512825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.514722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.516575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 196/385] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:17 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:17.522782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.660723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4b5f2-8e93-4712-9301-fddfd9f63d74 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.705866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32852dfc-9ed5-40ea-8982-79215bc7fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.706763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32852dfc-9ed5-40ea-8982-79215bc7fa98 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.707050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32852dfc-9ed5-40ea-8982-79215bc7fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.707328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32852dfc-9ed5-40ea-8982-79215bc7fa98 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.708552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32852dfc-9ed5-40ea-8982-79215bc7fa98 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.710073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32852dfc-9ed5-40ea-8982-79215bc7fa98 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:17.712380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 190/386] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:17 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:17.718921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:17.820089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c789b9-fca8-4681-b411-ad861127cfd7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.820380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c789b9-fca8-4681-b411-ad861127cfd7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.820559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c789b9-fca8-4681-b411-ad861127cfd7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.822285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 191/387] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:17.827792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bc31d-f5d3-47a6-8eb7-a2d7a5589350 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.828178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bc31d-f5d3-47a6-8eb7-a2d7a5589350 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.828468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bc31d-f5d3-47a6-8eb7-a2d7a5589350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.859141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f71bc31d-f5d3-47a6-8eb7-a2d7a5589350 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:17.860384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f71bc31d-f5d3-47a6-8eb7-a2d7a5589350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:17.928542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:17.936155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d52ab-11a8-4e36-9f1b-15a306802fd4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:17.988129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1c393-7651-484d-baf4-5c3d45f6ade8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:17.989034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1c393-7651-484d-baf4-5c3d45f6ade8 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:17.989318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1c393-7651-484d-baf4-5c3d45f6ade8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:17.989597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1c393-7651-484d-baf4-5c3d45f6ade8 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a/groups/5fe93cde2d554c178f32b06d537a614e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:17.991695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1c393-7651-484d-baf4-5c3d45f6ade8 None admin] RBAC: Authorizing `identity:create_grant(project_id=caaea6a33f9840079019e777b132555a, group_id=5fe93cde2d554c178f32b06d537a614e, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:17.993263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a1c393-7651-484d-baf4-5c3d45f6ade8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.001036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 192/388] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:18.016430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 197/389] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 15:59:17 2022] PUT /identity/v3/projects/caaea6a33f9840079019e777b132555a/groups/5fe93cde2d554c178f32b06d537a614e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:18.025427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bc31d-f5d3-47a6-8eb7-a2d7a5589350 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.147184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bc31d-f5d3-47a6-8eb7-a2d7a5589350 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.158016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09213d-dd10-43be-804e-b5b832199e2c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.158848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09213d-dd10-43be-804e-b5b832199e2c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.159117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09213d-dd10-43be-804e-b5b832199e2c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.160536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 198/390] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:18.166641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c77e0-5cc0-48a1-91fc-1b4b35c00623 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.166940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c77e0-5cc0-48a1-91fc-1b4b35c00623 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.167220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c77e0-5cc0-48a1-91fc-1b4b35c00623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.188063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9f4c77e0-5cc0-48a1-91fc-1b4b35c00623 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:18.189442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f4c77e0-5cc0-48a1-91fc-1b4b35c00623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:18.193103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.193986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.194275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.194562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.221747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:18.230109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:18.231058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:18.231830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:18.233151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 193/391] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:18.242101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.247366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.287409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.332761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 199/392] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:18.358820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c77e0-5cc0-48a1-91fc-1b4b35c00623 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.376170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979285d-ce76-4c9f-a088-6d91953e13e9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.416575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.417533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.418034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.418504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.419442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:18.420178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.421796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.432442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 194/393] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:18.438793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.501111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c77e0-5cc0-48a1-91fc-1b4b35c00623 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.562479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.563674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.564292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.565402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.587841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:18.596714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:18.603458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:18.604345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:18.605161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:18.611531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 200/394] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:18.619732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.626833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3786c-e1e5-4000-9404-34008a313050 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.650855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:18.698508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.699359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.699647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.699981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.701211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.703068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.707372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 195/395] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:18.710509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.813986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5fd687-cc1f-478b-a095-6226b390bd88 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.833019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4578981-e856-4acd-93ba-197afb2403e9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:18.869132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.870300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.870596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.870931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.872200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.873668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.875918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 196/396] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:18.880812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:18.882005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.883513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:18.883804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:18.884090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:18.884850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:18.885165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:18.886569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:18.912491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 201/397] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:18.913963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:18.926258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.014574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40d2d-efc5-4a24-8f90-f311514f4f96 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.054796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.055698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.055980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.056260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a/groups/5fe93cde2d554c178f32b06d537a614e/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.058585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] RBAC: Authorizing `identity:create_grant(project_id=caaea6a33f9840079019e777b132555a, group_id=5fe93cde2d554c178f32b06d537a614e, role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.060827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.078966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 197/398] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 15:59:18 2022] PUT /identity/v3/projects/caaea6a33f9840079019e777b132555a/groups/5fe93cde2d554c178f32b06d537a614e/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:19.105653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.137452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66692b50-7bf4-4c24-b2ab-f9604011df5c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.180042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa9363-7326-4cd8-8969-d5e33382c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.180921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa9363-7326-4cd8-8969-d5e33382c9ed None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.181255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa9363-7326-4cd8-8969-d5e33382c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.181588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa9363-7326-4cd8-8969-d5e33382c9ed None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.183288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa9363-7326-4cd8-8969-d5e33382c9ed None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.184838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa9363-7326-4cd8-8969-d5e33382c9ed None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.186588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 202/399] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:18 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:19.192366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.288160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3966-1873-4e47-be19-12c6f4499a89 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.288364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3966-1873-4e47-be19-12c6f4499a89 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.288644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3966-1873-4e47-be19-12c6f4499a89 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.290294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 203/400] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:19.301750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7ee70-d063-42d7-95a8-3a0b2268323e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.301750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7ee70-d063-42d7-95a8-3a0b2268323e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.301750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7ee70-d063-42d7-95a8-3a0b2268323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.319609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d3f7ee70-d063-42d7-95a8-3a0b2268323e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:19.320161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d3f7ee70-d063-42d7-95a8-3a0b2268323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:19.342219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.359166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da58e26-b747-439f-95dd-0b56edb90a15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.432247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.433233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.434998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.434998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.437264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.439318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.445436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 198/401] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:19.454246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.458272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 204/402] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:19.486168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7ee70-d063-42d7-95a8-3a0b2268323e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.584992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.612797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69982-60c1-47cb-a3dd-5d8e9de9de34 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.662406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7ee70-d063-42d7-95a8-3a0b2268323e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:19.665949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9896ac-c5b9-4263-97bc-9f2c37a79824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.666933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9896ac-c5b9-4263-97bc-9f2c37a79824 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.667221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9896ac-c5b9-4263-97bc-9f2c37a79824 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.667506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9896ac-c5b9-4263-97bc-9f2c37a79824 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/groups/5fe93cde2d554c178f32b06d537a614e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.669808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9896ac-c5b9-4263-97bc-9f2c37a79824 None admin] RBAC: Authorizing `identity:create_grant(project_id=51d1f730296e4ce69df77b2c3a336fc3, group_id=5fe93cde2d554c178f32b06d537a614e, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:19.671340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9896ac-c5b9-4263-97bc-9f2c37a79824 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:19.689087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 199/403] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 15:59:19 2022] PUT /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/groups/5fe93cde2d554c178f32b06d537a614e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:19.690839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5dee-62fa-4dbd-bafd-b59bc98a7cdd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.691186 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5dee-62fa-4dbd-bafd-b59bc98a7cdd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.691468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5dee-62fa-4dbd-bafd-b59bc98a7cdd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.693330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 200/404] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:19.708014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd92bf-95e7-45ce-b9e8-01c645800fc5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.708656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd92bf-95e7-45ce-b9e8-01c645800fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.708656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd92bf-95e7-45ce-b9e8-01c645800fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.733973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-04fd92bf-95e7-45ce-b9e8-01c645800fc5 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:19.735029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:19.735524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-04fd92bf-95e7-45ce-b9e8-01c645800fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:19.736356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:19.737402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:19.737927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:19.750382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:19.766626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:19.784374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:19.785345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:19.786191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:19.789045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 205/405] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:19.795298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:19.869017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 201/406] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 15:59:19.901810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd92bf-95e7-45ce-b9e8-01c645800fc5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.008670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd9d30-b4e2-4a55-be47-2336584bc9e1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.046786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.058358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.059052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.059341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.059703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.060443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:20.060768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.062235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.072326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 206/407] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:20.077967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.110953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd92bf-95e7-45ce-b9e8-01c645800fc5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.166347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.167296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.167682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.168061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.184755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:20.196627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:20.197639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:20.198472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:20.199096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 202/408] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:20.204385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.207750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.209514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebbde2-4544-4ca1-9de5-52e94202de2b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.253749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.255295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.255874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.256425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.258159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.260147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.262818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 207/409] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:20.269776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.341313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.353764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423bc5-bcdd-485e-b5de-581ca69f56ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.401509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.401971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.402265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.402548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.403692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:20.404153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.405638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.408349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8b8c3-a356-40a3-9f75-8784d868f3c1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.418024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 203/410] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:20.423565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.474045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.475989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.476724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.477394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.481401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.484509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.489218 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 208/411] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:20.499906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.514486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.579165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cef3-9dad-4d26-a7e0-b0a9515ab018 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.634492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.636090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.636444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.637118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] PATH_INFO: `/v3/groups/961912e2a2c644b9b5226d1397c8f5bb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.638374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] RBAC: Authorizing `identity:get_group(group_id=961912e2a2c644b9b5226d1397c8f5bb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.640536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.642606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 204/412] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/groups/961912e2a2c644b9b5226d1397c8f5bb => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:20.651189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:20.697359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffb6f0-fe5f-4a9d-b037-4aca2675b30c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.757812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.770179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.772165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.773013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.773804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/groups/5fe93cde2d554c178f32b06d537a614e/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.778843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=51d1f730296e4ce69df77b2c3a336fc3, group_id=5fe93cde2d554c178f32b06d537a614e, role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.781991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.809145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 209/413] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 15:59:20 2022] PUT /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3/groups/5fe93cde2d554c178f32b06d537a614e/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:59:20.842649 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039ff1e-baa0-45bf-9d65-0178c066896e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:20.882236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:20.889163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0e44f-962c-428f-bdef-7e37561be3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:20.890478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e44f-962c-428f-bdef-7e37561be3fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:20.891153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e44f-962c-428f-bdef-7e37561be3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:20.891444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e44f-962c-428f-bdef-7e37561be3fe None admin] PATH_INFO: `/v3/projects/a913cabdc8c6449f971e776cd4ba8813` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:20.892678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e44f-962c-428f-bdef-7e37561be3fe None admin] RBAC: Authorizing `identity:get_project(project_id=a913cabdc8c6449f971e776cd4ba8813)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:20.894257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e44f-962c-428f-bdef-7e37561be3fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:20.896956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 205/414] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:20 2022] GET /identity/v3/projects/a913cabdc8c6449f971e776cd4ba8813 => generated 344 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:20.903296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.122956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf105f-bd68-47a7-9bd7-f051a1f65fdd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.189485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a7e99-e5e2-41f2-a427-8a3b84373b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.190785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a7e99-e5e2-41f2-a427-8a3b84373b8a None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.191294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a7e99-e5e2-41f2-a427-8a3b84373b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.191808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a7e99-e5e2-41f2-a427-8a3b84373b8a None admin] PATH_INFO: `/v3/projects/a913cabdc8c6449f971e776cd4ba8813/groups/961912e2a2c644b9b5226d1397c8f5bb/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.195336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a7e99-e5e2-41f2-a427-8a3b84373b8a None admin] RBAC: Authorizing `identity:create_grant(project_id=a913cabdc8c6449f971e776cd4ba8813, group_id=961912e2a2c644b9b5226d1397c8f5bb, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.196850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a7e99-e5e2-41f2-a427-8a3b84373b8a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:21.216492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 210/415] 10.222.0.88 () {58 vars in 1442 bytes} [Thu Jun 16 15:59:20 2022] PUT /identity/v3/projects/a913cabdc8c6449f971e776cd4ba8813/groups/961912e2a2c644b9b5226d1397c8f5bb/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 15:59:21.350152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9448d1-b4d9-4899-bb5a-1c1dc5cf8063 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.350657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9448d1-b4d9-4899-bb5a-1c1dc5cf8063 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.351222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9448d1-b4d9-4899-bb5a-1c1dc5cf8063 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.353757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 206/416] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:21.360411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2266d1-bfbf-4f38-b2b3-03b049d2e398 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.360702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2266d1-bfbf-4f38-b2b3-03b049d2e398 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.360985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2266d1-bfbf-4f38-b2b3-03b049d2e398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.388973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ca2266d1-bfbf-4f38-b2b3-03b049d2e398 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:21.391444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca2266d1-bfbf-4f38-b2b3-03b049d2e398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:21.456378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:21.497828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 211/417] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:21.527903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9448d1-b4d9-4899-bb5a-1c1dc5cf8063 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.666326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9448d1-b4d9-4899-bb5a-1c1dc5cf8063 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.712255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.714432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.715251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.715981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.731969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:21.743588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:21.744887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:21.746298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:21.753840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 207/418] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:21.759414 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2266d1-bfbf-4f38-b2b3-03b049d2e398 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:21.932606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2266d1-bfbf-4f38-b2b3-03b049d2e398 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:21.992491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:21.993551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:21.994083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:21.994569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:21.995507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:21.996012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:21.997559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:22.009825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 212/419] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:22.012803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:22.111029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:22.178868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadf607-7d3d-4d38-8bb3-fdec5f390b81 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:22.259023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff4efc-8a4a-48bc-afbd-dd651b7bb797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:22.259703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4efc-8a4a-48bc-afbd-dd651b7bb797 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.259975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4efc-8a4a-48bc-afbd-dd651b7bb797 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.260814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4efc-8a4a-48bc-afbd-dd651b7bb797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.443945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff4efc-8a4a-48bc-afbd-dd651b7bb797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:22.445855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff4efc-8a4a-48bc-afbd-dd651b7bb797 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:22.544768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:22.629265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 208/420] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:22.631826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:22.721457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:22.746620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07634a03-061d-471b-a610-92afb19dafb6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.747265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07634a03-061d-471b-a610-92afb19dafb6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.747754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07634a03-061d-471b-a610-92afb19dafb6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.749921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 209/421] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:22.759368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed3a34-422d-4cb3-a6ab-f0ede52cbeab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.760144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed3a34-422d-4cb3-a6ab-f0ede52cbeab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.760634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed3a34-422d-4cb3-a6ab-f0ede52cbeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.795473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15ed3a34-422d-4cb3-a6ab-f0ede52cbeab None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:22.797559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15ed3a34-422d-4cb3-a6ab-f0ede52cbeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:22.807864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7154de-b936-4d43-b8d0-b6c7c6ec3cb1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:22.859892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d2d27-3890-4692-9a03-963e40b9b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:22.860864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d2d27-3890-4692-9a03-963e40b9b49f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.861142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d2d27-3890-4692-9a03-963e40b9b49f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.861615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d2d27-3890-4692-9a03-963e40b9b49f None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.863488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d2d27-3890-4692-9a03-963e40b9b49f None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:22.865055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d2d27-3890-4692-9a03-963e40b9b49f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:22.867209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 213/422] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:22 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:22.868639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607a36-9d2f-4471-8ec0-89dec44a283b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.869191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607a36-9d2f-4471-8ec0-89dec44a283b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.869664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607a36-9d2f-4471-8ec0-89dec44a283b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.871161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 214/423] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:22.876846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cd3bb-68eb-4afd-b61d-93c91a3db331 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:22.877339 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cd3bb-68eb-4afd-b61d-93c91a3db331 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:22.877898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cd3bb-68eb-4afd-b61d-93c91a3db331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:22.892292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 210/424] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:22.893956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed3a34-422d-4cb3-a6ab-f0ede52cbeab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:22.896543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ec0cd3bb-68eb-4afd-b61d-93c91a3db331 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:22.898104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ec0cd3bb-68eb-4afd-b61d-93c91a3db331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:22.986616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 215/425] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:22.994232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cd3bb-68eb-4afd-b61d-93c91a3db331 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.018988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.030274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed3a34-422d-4cb3-a6ab-f0ede52cbeab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.061840 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.069248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.070527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.071014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.071490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.073078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.074827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.081888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 211/426] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:22 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.083143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.136560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cd3bb-68eb-4afd-b61d-93c91a3db331 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.182280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.183114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.183916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.184216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.197370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed99b0-af95-464e-84ea-2a148df63b6d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.198534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:23.208979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:23.209905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:23.211143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 216/427] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:23.211966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.242929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.243731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.244041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.244290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.260804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:23.270130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:23.271038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 15:59:23.271878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:23.276206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 212/428] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:23.277112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.298028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.356025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.360067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65965ab-d8c4-4c95-be85-29d84efee9e9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.391492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29eb35-654c-4d82-a82d-f6eb294cf429 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.407149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.408407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.409676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.410265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.411655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=caaea6a33f9840079019e777b132555a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:23.412172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.414407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.423714 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 217/429] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.425057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.438158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.438844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.439132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.439412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.440473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:23.440610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.450192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.467183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 213/430] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.472837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.605704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25131e19-2e85-421e-bf5e-8811c464a812 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.617221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.641380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17472b57-3b64-4f3e-9174-d15ca142e4b7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.680460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8f57d-003e-4c0c-bb21-6e8eae18fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.681472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8f57d-003e-4c0c-bb21-6e8eae18fed8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.681996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8f57d-003e-4c0c-bb21-6e8eae18fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.682497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8f57d-003e-4c0c-bb21-6e8eae18fed8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.685420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.686162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.686450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.686730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.687491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:23.687799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.689552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.699497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 218/431] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:23.705140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.740000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.763092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8f57d-003e-4c0c-bb21-6e8eae18fed8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.765115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8f57d-003e-4c0c-bb21-6e8eae18fed8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:23.831057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2c23e-19aa-484a-a0bd-859000b4f4be None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:23.859019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:23.877060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515f666-f915-4732-8ae4-68932bc1b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:23.878277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515f666-f915-4732-8ae4-68932bc1b7bf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.878589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515f666-f915-4732-8ae4-68932bc1b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.879050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515f666-f915-4732-8ae4-68932bc1b7bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.882645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 214/432] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:23.884061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424b94-1219-432f-a9e8-51a57146d585 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:23.884354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424b94-1219-432f-a9e8-51a57146d585 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:23.884639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424b94-1219-432f-a9e8-51a57146d585 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:23.886295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 215/433] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:23.889339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424b94-1219-432f-a9e8-51a57146d585 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:23.971424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515f666-f915-4732-8ae4-68932bc1b7bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:23.973155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515f666-f915-4732-8ae4-68932bc1b7bf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.010870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.024462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424b94-1219-432f-a9e8-51a57146d585 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.072222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98341c-40aa-49ef-8021-aed1e42f0955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.073095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98341c-40aa-49ef-8021-aed1e42f0955 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.073377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98341c-40aa-49ef-8021-aed1e42f0955 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.073652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98341c-40aa-49ef-8021-aed1e42f0955 None admin] PATH_INFO: `/v3/projects/caaea6a33f9840079019e777b132555a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.074934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98341c-40aa-49ef-8021-aed1e42f0955 None admin] RBAC: Authorizing `identity:get_project(project_id=caaea6a33f9840079019e777b132555a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.078351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98341c-40aa-49ef-8021-aed1e42f0955 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.078484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 216/434] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/projects/caaea6a33f9840079019e777b132555a => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:24.079587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8754670-aa50-41eb-b7fc-c247feef5703 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.079875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8754670-aa50-41eb-b7fc-c247feef5703 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.080153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8754670-aa50-41eb-b7fc-c247feef5703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.102401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d8754670-aa50-41eb-b7fc-c247feef5703 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:24.102401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8754670-aa50-41eb-b7fc-c247feef5703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:24.113366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 219/435] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:59:24.114811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77866d-5190-473a-a2da-ca170b6b407b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.115099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77866d-5190-473a-a2da-ca170b6b407b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.115381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef77866d-5190-473a-a2da-ca170b6b407b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.116818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 220/436] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:24.117525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77866d-5190-473a-a2da-ca170b6b407b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.157360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.182772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 217/437] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:24.183610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8754670-aa50-41eb-b7fc-c247feef5703 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.229499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef77866d-5190-473a-a2da-ca170b6b407b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.255376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.264118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73a548-890b-4dde-9e41-c51423c6fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.264979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a548-890b-4dde-9e41-c51423c6fa96 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.265255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a548-890b-4dde-9e41-c51423c6fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.265529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73a548-890b-4dde-9e41-c51423c6fa96 None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.266783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73a548-890b-4dde-9e41-c51423c6fa96 None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.268713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73a548-890b-4dde-9e41-c51423c6fa96 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.270351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 221/438] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.271474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f9b22-2f29-44b6-b2db-d8c2a8061fe8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.271752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f9b22-2f29-44b6-b2db-d8c2a8061fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.272023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f9b22-2f29-44b6-b2db-d8c2a8061fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.288254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9f0f9b22-2f29-44b6-b2db-d8c2a8061fe8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:24.288643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8754670-aa50-41eb-b7fc-c247feef5703 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.289435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f0f9b22-2f29-44b6-b2db-d8c2a8061fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:24.322477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.323345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.323628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.323906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.325078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.326554 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.328230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 218/439] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.328980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.354858 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 222/440] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:24.358685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f9b22-2f29-44b6-b2db-d8c2a8061fe8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.380937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.435683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e3a20-9f9c-44fc-810a-1d4f569c3c62 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.460143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f9b22-2f29-44b6-b2db-d8c2a8061fe8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.469295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.470133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.470413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.470686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.481241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:24.484493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:24.489479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:24.490437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:24.491227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 219/441] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:24.491868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.498510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.499222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.499503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.499777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.500600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=caaea6a33f9840079019e777b132555a) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:24.500906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.502401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.508427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 223/442] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=caaea6a33f9840079019e777b132555a&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.509192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.615528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96641217-8889-4f2b-9941-b6622f44925d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.623881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.632265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6f321-5873-454f-b0bf-fe7eda4dc9fc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.656048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.657248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.657711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.658241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.659640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.661169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.667919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 220/443] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.667919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.678145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.679512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.679976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.680310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.692986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:24.700565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:24.701549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:24.702458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:24.703140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 224/444] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:24.704520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.757164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:24.794748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aed9c5-5342-4833-b05d-6abc4c34803d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.823761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac13d8-e0e0-47e6-ae02-c18bbfe791f6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:24.843886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.844649 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.844935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.848409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.848409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:24.848409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.848409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.858763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 221/445] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.860182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.870194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:24.870535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:24.870829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:24.871101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:24.872076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:24.872386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:24.873894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:24.880083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 225/446] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:24.880887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:24.917102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.001970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee929d-4aac-4a54-aed5-2a7821f5a144 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.008112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6511f-018c-492c-9250-1e81fd1d88cc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.023771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.041670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.042425 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.042726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.042985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.043742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.044056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.045468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.056456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 222/447] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:25.057277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.058055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.058341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.058620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.062124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.144012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.145477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.188327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.194060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10866f44-95fe-4289-baca-07080dca9229 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.239147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.239810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.240099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.240388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.262460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 226/448] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:25.267771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.282216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.317277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.318771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.379667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9212b-4adf-4abd-bd67-4f1f889acc21 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.412807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.414604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.415465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.415790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.416064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] PATH_INFO: `/v3/projects/51d1f730296e4ce69df77b2c3a336fc3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.417173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 223/449] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:59:25.417278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] RBAC: Authorizing `identity:get_project(project_id=51d1f730296e4ce69df77b2c3a336fc3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.418700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.420365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 227/450] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/projects/51d1f730296e4ce69df77b2c3a336fc3 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:25.422601 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.426096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.514589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.529354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b307a35-ea08-4f67-8906-0cbb007386e1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.534458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a9022-b883-457e-a436-885f963c50f1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.569167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.570089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.570360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.570633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] PATH_INFO: `/v3/groups/5fe93cde2d554c178f32b06d537a614e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.571791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] RBAC: Authorizing `identity:get_group(group_id=5fe93cde2d554c178f32b06d537a614e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.573216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.574866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 228/451] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/groups/5fe93cde2d554c178f32b06d537a614e => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:25.580506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.580713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.581376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.581658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.581983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] PATH_INFO: `/v3/projects/a913cabdc8c6449f971e776cd4ba8813` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.583178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] RBAC: Authorizing `identity:get_project(project_id=a913cabdc8c6449f971e776cd4ba8813)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.584612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.586363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 224/452] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/projects/a913cabdc8c6449f971e776cd4ba8813 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:25.591723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.656592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.684183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6966b6-2753-4759-ad48-65f9f1a2a039 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.691655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e01d61-91a0-4fc5-8347-76bc1be43f11 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.718175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.718842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.719132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.719415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.720249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] RBAC: Adding query filter params (group.id=5fe93cde2d554c178f32b06d537a614e, role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=51d1f730296e4ce69df77b2c3a336fc3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.720568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.722614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.726798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b43de-98b7-4940-87b2-796a78fece62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.727653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b43de-98b7-4940-87b2-796a78fece62 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.727932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b43de-98b7-4940-87b2-796a78fece62 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.728205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b43de-98b7-4940-87b2-796a78fece62 None admin] PATH_INFO: `/v3/groups/961912e2a2c644b9b5226d1397c8f5bb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.728470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 229/453] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/role_assignments?group.id=5fe93cde2d554c178f32b06d537a614e&scope.project.id=51d1f730296e4ce69df77b2c3a336fc3&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:25.729390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b43de-98b7-4940-87b2-796a78fece62 None admin] RBAC: Authorizing `identity:get_group(group_id=961912e2a2c644b9b5226d1397c8f5bb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.730854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b43de-98b7-4940-87b2-796a78fece62 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.732505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 225/454] 10.222.0.88 () {56 vars in 1180 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/groups/961912e2a2c644b9b5226d1397c8f5bb => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:25.739237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:25.755687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:25.849792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afdb14-f16f-4e65-ac5f-e1c0e7149daf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:25.885991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dc648-1329-4416-b901-bc46b4a20e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:25.886661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dc648-1329-4416-b901-bc46b4a20e10 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:25.886941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dc648-1329-4416-b901-bc46b4a20e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:25.887216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dc648-1329-4416-b901-bc46b4a20e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:25.888045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dc648-1329-4416-b901-bc46b4a20e10 None admin] RBAC: Adding query filter params (group.id=961912e2a2c644b9b5226d1397c8f5bb, role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=a913cabdc8c6449f971e776cd4ba8813) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:25.888354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dc648-1329-4416-b901-bc46b4a20e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:25.889939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dc648-1329-4416-b901-bc46b4a20e10 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:25.896137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 230/455] 10.222.0.88 () {56 vars in 1376 bytes} [Thu Jun 16 15:59:25 2022] GET /identity/v3/role_assignments?group.id=961912e2a2c644b9b5226d1397c8f5bb&scope.project.id=a913cabdc8c6449f971e776cd4ba8813&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:28.570640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af176c2e-8bac-404e-9e35-1cb32a537ec3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.571157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af176c2e-8bac-404e-9e35-1cb32a537ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.571451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af176c2e-8bac-404e-9e35-1cb32a537ec3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.572938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 226/456] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:28.578188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5a0d-ec53-4bfe-9f13-8b9be34e1bdb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.579069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5a0d-ec53-4bfe-9f13-8b9be34e1bdb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.579567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b5a0d-ec53-4bfe-9f13-8b9be34e1bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.603497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ce9b5a0d-ec53-4bfe-9f13-8b9be34e1bdb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:28.605027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ce9b5a0d-ec53-4bfe-9f13-8b9be34e1bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:28.695077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 231/457] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:28.732169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af176c2e-8bac-404e-9e35-1cb32a537ec3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:28.781314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36883f1-3e62-417c-b09d-cf574ff11f77 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.781985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36883f1-3e62-417c-b09d-cf574ff11f77 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.782489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36883f1-3e62-417c-b09d-cf574ff11f77 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.784086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 232/458] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:28.790752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9c08-79b7-400c-8872-3256ea672ead None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.791041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9c08-79b7-400c-8872-3256ea672ead None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.791327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9c08-79b7-400c-8872-3256ea672ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.814088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:28.819039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3e1c9c08-79b7-400c-8872-3256ea672ead None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:28.820746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3e1c9c08-79b7-400c-8872-3256ea672ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:28.859534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af176c2e-8bac-404e-9e35-1cb32a537ec3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:28.910363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:28.911165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.911450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.911728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.914654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:28.923878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 233/459] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:28.926019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a3a2-8ea6-472d-ac5f-5bc5dd3876bb None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.926517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a3a2-8ea6-472d-ac5f-5bc5dd3876bb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.926998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6a3a2-8ea6-472d-ac5f-5bc5dd3876bb None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.928800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 234/460] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:28.936923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68b122-923f-45bf-a06f-06f421488437 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:28.936923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68b122-923f-45bf-a06f-06f421488437 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:28.936923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68b122-923f-45bf-a06f-06f421488437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:28.937969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:28.939471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:28.964308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cd68b122-923f-45bf-a06f-06f421488437 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:28.964889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:28.965973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:28.967041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:28.967041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:28.967041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:28.967041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:28.967288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cd68b122-923f-45bf-a06f-06f421488437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:28.968805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 227/461] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:28.970056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.060264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 235/462] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:29.062290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68b122-923f-45bf-a06f-06f421488437 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.093213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e50465-269f-43b9-9dec-709da60fd085 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.129544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.138260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.138917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.139121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.158756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.163787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.167844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:29.192057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:29.193027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:29.193911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:29.193911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:29.193911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:29.193911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:29.201024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 228/463] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:29.202294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.207355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68b122-923f-45bf-a06f-06f421488437 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.245187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.249251 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.249960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.250249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.250593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.251325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.251644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.253554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.265493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 236/464] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:29.266342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.316595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863eb10-dd1b-4432-a2e5-55b71d894958 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.351869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df3492-e2b9-4487-a845-af15366c5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.352591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3492-e2b9-4487-a845-af15366c5b14 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.352871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3492-e2b9-4487-a845-af15366c5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.353198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df3492-e2b9-4487-a845-af15366c5b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.363764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df3492-e2b9-4487-a845-af15366c5b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.365234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df3492-e2b9-4487-a845-af15366c5b14 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.379711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68471e95-c04c-4413-a959-62d1a73a3f29 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-43df3492-e2b9-4487-a845-af15366c5b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:29.386536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:29.387427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:29.388325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:29.388325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:29.388325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:29.388325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:29.388325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 229/465] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:29.389110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435fc16-c8d4-490b-8bc3-6702e278a089 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.389317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435fc16-c8d4-490b-8bc3-6702e278a089 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.389594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435fc16-c8d4-490b-8bc3-6702e278a089 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.391080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 230/466] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:29.396071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435fc16-c8d4-490b-8bc3-6702e278a089 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.416488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.428271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.428960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.429248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.429532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.430307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.430619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.432013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.451163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 237/467] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:29.451163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.501119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.514114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435fc16-c8d4-490b-8bc3-6702e278a089 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.549685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d574de3-e8c2-4c13-88e2-5b6e36fd97e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.550411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d574de3-e8c2-4c13-88e2-5b6e36fd97e2 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.550766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d574de3-e8c2-4c13-88e2-5b6e36fd97e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.551051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d574de3-e8c2-4c13-88e2-5b6e36fd97e2 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.551759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d574de3-e8c2-4c13-88e2-5b6e36fd97e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.553201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d574de3-e8c2-4c13-88e2-5b6e36fd97e2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.557041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdffa0a-216c-4532-91d3-a771fdcfb5fa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.574072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 231/468] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:29 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:29.575388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8fcac-35c0-48d4-a29e-a910f49ba16d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.575680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8fcac-35c0-48d4-a29e-a910f49ba16d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.575962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8fcac-35c0-48d4-a29e-a910f49ba16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.599917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b5c8fcac-35c0-48d4-a29e-a910f49ba16d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:29.601587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5c8fcac-35c0-48d4-a29e-a910f49ba16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:29.610637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.611440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.611579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.611958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.613044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:29.613349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.614808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.628724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 238/469] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:29.634097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.696367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.736563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 232/470] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:29.738637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8fcac-35c0-48d4-a29e-a910f49ba16d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.762818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2955b7c-94a1-48be-ae7d-4dab5a2862c7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.801986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.802678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.802962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.803238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.803948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.805791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.830515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 239/471] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:29 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:29.831731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:29.860866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:29.897041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8fcac-35c0-48d4-a29e-a910f49ba16d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.942752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30508cfa-a82d-4620-8dc8-c24993a0a197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:29.943920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508cfa-a82d-4620-8dc8-c24993a0a197 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:29.944462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508cfa-a82d-4620-8dc8-c24993a0a197 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:29.944944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508cfa-a82d-4620-8dc8-c24993a0a197 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:29.945940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30508cfa-a82d-4620-8dc8-c24993a0a197 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:29.947699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30508cfa-a82d-4620-8dc8-c24993a0a197 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:29.975041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 233/472] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:29 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:29.975450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2510cd-9e8d-48aa-a57f-fde4687decaf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:29.977314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.010780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed3ff0-1f65-43b9-b97e-59a82db00ae0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.011424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed3ff0-1f65-43b9-b97e-59a82db00ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.011914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed3ff0-1f65-43b9-b97e-59a82db00ae0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.014923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 234/473] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:30.019463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4b38-85ae-4b45-a36c-41aad403e8cc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.020076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4b38-85ae-4b45-a36c-41aad403e8cc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.020359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af4b38-85ae-4b45-a36c-41aad403e8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.024539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.025446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.025709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.026123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.042716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.044193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.047648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-07af4b38-85ae-4b45-a36c-41aad403e8cc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:30.049478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07af4b38-85ae-4b45-a36c-41aad403e8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:30.071570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:30.072573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:30.073403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:30.073403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:30.073403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:30.073403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:30.075776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 240/474] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:30.085266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.176949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 235/475] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:30.212412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4b38-85ae-4b45-a36c-41aad403e8cc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.224341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f0263-aaaf-42e9-9f4c-82d228840770 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.264957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.265658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.265982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.266264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.266993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:30.267671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.268750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.283393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 241/476] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:30.288701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.298815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.318200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.354790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af4b38-85ae-4b45-a36c-41aad403e8cc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.404138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.404752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.405188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.405481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.407150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.408586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.417402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 15:59:30.417402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 15:59:30.417402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [parameters: {'id': 'fa6c8288364b4b6080cedcfdf4e3aa20', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 15:59:30.417402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jun 16 15:59:30.440174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:59:30.441179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 15:59:30.442143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 15:59:30.442988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 15:59:30.443912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 15:59:30.444719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': 'fa6c8288364b4b6080cedcfdf4e3aa20', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:30.445504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 15:59:30.446369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:30.447200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 236/477] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 15:59:30 2022] POST /identity/v3/roles => generated 144 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:30.447273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfcd2e-454e-4193-93ff-22624736d0e7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.458021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.509852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.510845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.511296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.511714 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.512958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.515229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.554518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 242/478] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:30 2022] POST /identity/v3/users => generated 249 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 15:59:30.626392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953852a-5687-45c3-ae36-63932a4f8b5f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.656683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.667773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf0c69-b94e-4f89-956d-44dbef59f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.668962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf0c69-b94e-4f89-956d-44dbef59f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.669451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf0c69-b94e-4f89-956d-44dbef59f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.669914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf0c69-b94e-4f89-956d-44dbef59f8f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.682514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf0c69-b94e-4f89-956d-44dbef59f8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2dbf0c69-b94e-4f89-956d-44dbef59f8f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:30.691709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:30.692809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 15:59:30.693833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:30.696412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 237/479] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 15:59:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:30.702315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:30.799761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:30.843795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4c243-4e51-4855-832c-4557a825066a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:30.882223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dc9c4-dfd2-42f6-ba5b-deaac0207d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:30.883019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dc9c4-dfd2-42f6-ba5b-deaac0207d60 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:30.883399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dc9c4-dfd2-42f6-ba5b-deaac0207d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:30.883756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dc9c4-dfd2-42f6-ba5b-deaac0207d60 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:30.884588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dc9c4-dfd2-42f6-ba5b-deaac0207d60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:30.884966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dc9c4-dfd2-42f6-ba5b-deaac0207d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:30.886523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dc9c4-dfd2-42f6-ba5b-deaac0207d60 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:30.895733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 243/480] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:32.422848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e082-f347-45e1-90f3-ef86681a2434 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.424061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e082-f347-45e1-90f3-ef86681a2434 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.424705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e082-f347-45e1-90f3-ef86681a2434 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.426793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 238/481] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:32.431865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc129433-1387-4b25-8f98-023430091a13 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.432279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc129433-1387-4b25-8f98-023430091a13 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.432448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc129433-1387-4b25-8f98-023430091a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.458816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cc129433-1387-4b25-8f98-023430091a13 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:32.459767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cc129433-1387-4b25-8f98-023430091a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:32.553157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 244/482] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:32.592858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7e082-f347-45e1-90f3-ef86681a2434 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:32.690655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd9312-f4ee-460e-bfec-be8a3c94bdfd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.690979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd9312-f4ee-460e-bfec-be8a3c94bdfd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.691164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd9312-f4ee-460e-bfec-be8a3c94bdfd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.692815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 245/483] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:32.698031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defec1e5-3fd8-45f1-800f-d6b1c4d0b163 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.698322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defec1e5-3fd8-45f1-800f-d6b1c4d0b163 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.698606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defec1e5-3fd8-45f1-800f-d6b1c4d0b163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.717075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-defec1e5-3fd8-45f1-800f-d6b1c4d0b163 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:32.718296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-defec1e5-3fd8-45f1-800f-d6b1c4d0b163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:32.751999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7e082-f347-45e1-90f3-ef86681a2434 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:32.781289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:32.793844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 246/484] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:32.818241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defec1e5-3fd8-45f1-800f-d6b1c4d0b163 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:32.830490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:32.831389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.831694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.831994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.852687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:32.870841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:32.871911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:32.872815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 239/485] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:32.878936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:32.934750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defec1e5-3fd8-45f1-800f-d6b1c4d0b163 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:32.955174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:32.979760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab9f10-9a8b-454c-bd22-a3df6104fbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:32.981258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab9f10-9a8b-454c-bd22-a3df6104fbfb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:32.981829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab9f10-9a8b-454c-bd22-a3df6104fbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:32.982310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab9f10-9a8b-454c-bd22-a3df6104fbfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:32.995504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab9f10-9a8b-454c-bd22-a3df6104fbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-fbab9f10-9a8b-454c-bd22-a3df6104fbfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:33.004158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:33.005065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:33.005881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:33.008520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 247/486] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:33.014073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a260dd-39da-497c-b568-190546aceabf None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.014612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a260dd-39da-497c-b568-190546aceabf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.015138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a260dd-39da-497c-b568-190546aceabf None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.016964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 248/487] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:33.022409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a260dd-39da-497c-b568-190546aceabf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.035323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea424feb-81cb-48d7-9f5d-8c687f0497c3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.084097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.089631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b9dc4-479c-4147-b1c4-7e0961174a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.090643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b9dc4-479c-4147-b1c4-7e0961174a09 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.091345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b9dc4-479c-4147-b1c4-7e0961174a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.091776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b9dc4-479c-4147-b1c4-7e0961174a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.093224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b9dc4-479c-4147-b1c4-7e0961174a09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:33.093811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b9dc4-479c-4147-b1c4-7e0961174a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.096041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b9dc4-479c-4147-b1c4-7e0961174a09 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.108412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 240/488] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:33.109973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9eef2-f45b-4775-b8fb-11fa77fcf75a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.110411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9eef2-f45b-4775-b8fb-11fa77fcf75a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.110837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9eef2-f45b-4775-b8fb-11fa77fcf75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.131757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10a9eef2-f45b-4775-b8fb-11fa77fcf75a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:33.133181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10a9eef2-f45b-4775-b8fb-11fa77fcf75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:33.155681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a260dd-39da-497c-b568-190546aceabf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.195034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.195863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.196254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.196637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.197502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:33.197945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.199433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.214198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 249/489] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:33.215181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.224900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 241/490] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:33.226390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9eef2-f45b-4775-b8fb-11fa77fcf75a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.297700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.352902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c1edb-78a9-4bf3-9826-eb95b931ea67 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.372903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.381618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9eef2-f45b-4775-b8fb-11fa77fcf75a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.404557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.405641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.405958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.406260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.421972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.424350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.430032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.430947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.431295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.431645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.447055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:33.447838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:33.450185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:33.452017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:33.452017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:33.452017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:33.452017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:33.452017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 250/491] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:33.452522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.452604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:33.474929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:33.476246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:33.477290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:33.477290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:33.477290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:33.477290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:33.477290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 242/492] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:33.478054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.601439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f13ae-2893-4463-8ead-f7b320245231 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.604519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.618471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7021c7-f58b-4ef7-83d5-62dc06e68cf1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.646244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10035f53-2396-4c3f-94f7-cfe664dbe846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.647048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10035f53-2396-4c3f-94f7-cfe664dbe846 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.647332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10035f53-2396-4c3f-94f7-cfe664dbe846 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.647607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10035f53-2396-4c3f-94f7-cfe664dbe846 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.659088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10035f53-2396-4c3f-94f7-cfe664dbe846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:33.663235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.663898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.664178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.664455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.665175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:33.665482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-10035f53-2396-4c3f-94f7-cfe664dbe846 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:33.666870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:33.667928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:33.669375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:33.670019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.670100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 251/493] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:33.670100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476869-8762-4fc2-8c33-dcd297c2a4c3 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.670100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476869-8762-4fc2-8c33-dcd297c2a4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.670281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec476869-8762-4fc2-8c33-dcd297c2a4c3 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.673762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 252/494] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:33.673762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec476869-8762-4fc2-8c33-dcd297c2a4c3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.691561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 243/495] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:33.692646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.721567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.814150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec476869-8762-4fc2-8c33-dcd297c2a4c3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.819088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554deb7-b50a-4544-91a8-33e065c7638e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:33.849498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b23c22-9770-4c84-8625-e4fd7fcd0ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.850194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b23c22-9770-4c84-8625-e4fd7fcd0ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.850485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b23c22-9770-4c84-8625-e4fd7fcd0ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.850764 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b23c22-9770-4c84-8625-e4fd7fcd0ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.851487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b23c22-9770-4c84-8625-e4fd7fcd0ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:33.851796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b23c22-9770-4c84-8625-e4fd7fcd0ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.853168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b23c22-9770-4c84-8625-e4fd7fcd0ee3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.853261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:33.853931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.854221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.854510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.855257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:33.855574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:33.856955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:33.865416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 253/496] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:33.870187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 244/497] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:33.870187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.870383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7d5f-c87b-4834-b692-9bfcad96dbe4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.870383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7d5f-c87b-4834-b692-9bfcad96dbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.870383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7d5f-c87b-4834-b692-9bfcad96dbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.886213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.886387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-11df7d5f-c87b-4834-b692-9bfcad96dbe4 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:33.887424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-11df7d5f-c87b-4834-b692-9bfcad96dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:33.954048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 254/498] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:33.955372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfaec0-b8c1-4e24-8781-dd322ed21ca4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:33.955657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfaec0-b8c1-4e24-8781-dd322ed21ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:33.955934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfaec0-b8c1-4e24-8781-dd322ed21ca4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:33.957728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 255/499] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:33.958428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfaec0-b8c1-4e24-8781-dd322ed21ca4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:33.979045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:33.990274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7e543-739b-46cb-b397-c3982a592b52 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.025641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.026469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.026754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.027047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.035575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.037151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:34.054456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.055409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:34.056299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 245/500] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.057387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.074765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfaec0-b8c1-4e24-8781-dd322ed21ca4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.111748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4c5af-5a59-4597-9b04-0f4a9b0d294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.112564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c5af-5a59-4597-9b04-0f4a9b0d294d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.112835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c5af-5a59-4597-9b04-0f4a9b0d294d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.113111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c5af-5a59-4597-9b04-0f4a9b0d294d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.124664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4c5af-5a59-4597-9b04-0f4a9b0d294d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.126506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4c5af-5a59-4597-9b04-0f4a9b0d294d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.142390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e3b4c5af-5a59-4597-9b04-0f4a9b0d294d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:34.147933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.148888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.149760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:34.149760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:34.149760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:34.149760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:34.149760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 256/501] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.150706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e27a-3074-4fb5-b359-b5f8551e491c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.150994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e27a-3074-4fb5-b359-b5f8551e491c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.151275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e27a-3074-4fb5-b359-b5f8551e491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.169753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1c24e27a-3074-4fb5-b359-b5f8551e491c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:34.170982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1c24e27a-3074-4fb5-b359-b5f8551e491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:34.178776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a7328-9c99-46bc-9802-0711fcecbe1b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.215065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.215868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.216145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.216415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.225342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.226849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.236674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:34.241499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.242514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:34.243434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:34.243937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 257/502] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:34.244008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 246/503] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.244072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e27a-3074-4fb5-b359-b5f8551e491c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.244137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.353489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e27a-3074-4fb5-b359-b5f8551e491c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.369793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6742afa-9b01-49b1-b409-61f260661492 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.389360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.390186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.390466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.390745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.401506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.401653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:34.407449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.408120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.408409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.408693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:34.409329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.411512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:34.413247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:34.414512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.414512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.414512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.415922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 258/504] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.416734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.448406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 247/505] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:34.449221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.483956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.527196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd1f81-7063-4be6-897b-6f61e689a0d5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.560461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.561127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.561477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.561785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.562505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.562815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.564192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.577391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137388d9-5b88-4103-91ea-6973f29f51ef None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.578340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 259/506] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:34.579132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.613081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.613677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.614014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.614297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.615131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.615445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.616895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.634282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 248/507] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:34.635076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.679758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb486c-023c-4775-bf79-70466d4290a1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.682917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.697069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.714324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.715107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.715384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.715651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.725795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.727270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.734986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca1d7c-6192-4ef9-96fd-9f66bcae6d39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:34.745059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.746199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.747050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:34.747050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:34.747050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:34.747050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:34.747050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 260/508] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.747050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.775392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.776164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.776455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.776736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.777479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:34.777826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.779218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.787290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 249/509] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:34.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:34.856190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22895fa0-a1ac-445b-863b-3297b7824b5a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.891625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.892205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.892493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.892778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.902648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe65a7-cec8-404c-936a-1c5500507388 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:34.913529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.932157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:34.938597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:34.939390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:34.939673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:34.939947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:34.947861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.949295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.960804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:34.962301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:34.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:34.964782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:34.965681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 250/510] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:34.966270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.063836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 261/511] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:35.064671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.065334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cae343-ab9a-42f1-8a6d-4be55603708d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.102850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.103508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.103794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.104077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.140070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.144766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.171833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98531a5c-b34a-4031-b21c-9f867cfd87be None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.173482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.175026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.205707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.206448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.206733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.207013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.207732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:35.208040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.209412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.221004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 262/512] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:35.221835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.281433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 251/513] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:35.282306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.327071 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7383d-a25e-408f-ae84-0f81c53d81d9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.356469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.357053 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.365066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.366002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.366292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.366569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.377290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.378851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.392530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81a29b-7c96-4421-9e9c-35d3b9c9713a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:35.395809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:35.396822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:35.397688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:35.397688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:35.397688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:35.397688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:35.397688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 263/514] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:35.397688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.429513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.430269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.430561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.430841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.431678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:35.431992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.433446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.450583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 252/515] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:35.451396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.515364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672873-6c88-45ba-8ed2-e7e5de8bd4be None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.549707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.550527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.550814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.551093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.561296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f70cf-5a16-44d2-9b49-9a1429627b0a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.563978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.565512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:35.583063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:35.583996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:35.584857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:35.584857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:35.584857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:35.584857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:35.584857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 264/516] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:35.585128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.605859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.615651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.616320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.616612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.616884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.617579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.619045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.622453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.639150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 253/517] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:35 2022] POST /identity/v3/users => generated 247 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:35.640224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.713296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c6566-f900-4b1d-be6c-2a3dbb8303e9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.751161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.751975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.752260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.752547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.762737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f8f83-6455-487b-8e26-49483103360c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.765701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.767268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:35.786034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:35.786984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:35.788038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:35.788038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:35.788038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:35.788038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:35.792099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 265/518] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:35.793065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.812737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.813932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.813932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.814218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:35.814752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:35.815061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:35.817076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:35.830872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 254/519] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:35.832230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:35.901997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.916480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:35.936100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2980d5-c362-4e27-9c13-fe0a96e33613 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.965367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537d69a-9874-4460-b6e7-7fe811da29e1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:35.978114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:35.978772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:35.979056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:35.979335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.007520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.008794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.009079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.009620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.010365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:36.010831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.012819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.027271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 255/520] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:36.028118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.063825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.065307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.140696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a72b15-ba11-4ac4-9b85-6e8765c31046 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.143720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.170685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.185912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.186597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.186880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.187159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.187880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:36.188190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.189273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 266/521] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:36.189800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.190223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.208514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 256/522] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:36.209266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.323842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d05c1-a452-4015-a475-e69b05b222e7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.324208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8aee49-569e-4aa2-a00a-54385074ce84 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.363702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.364825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.365107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.365383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.373024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.374204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.374726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.375009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.375422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.376590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.384116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.385908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:36.395074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:36.395980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:36.397037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 267/523] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:36.397615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:36.404236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:36.405279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:36.406327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:36.407204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 257/524] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:36.408033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.437467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.455000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.514921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9b996-aa38-4cf8-aac3-1586eaa9d0dd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.524922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97f993-09b7-4cac-8b45-1dbc57c69d55 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.554505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.555307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.555587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.555858 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.565620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.566791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.567060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.567367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.567674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.568280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.575110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.576509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:36.587052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:36.587950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:36.593789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:36.593789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:36.593789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:36.593789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:36.593789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 268/525] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:36.593789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:36.594130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:36.594861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:36.595764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 258/526] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:36.601420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.692311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.701305 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.705852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65824b0-67ad-42d1-92b1-253257e470d8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.713248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c8df8-42af-4ad0-94c4-021b48d4a7e7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.744367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.745326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.745737 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.746182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.747031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:36.747348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.748806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.753143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.754591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.755083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.755507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.756452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:36.756928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.758578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.779346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 259/527] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:36.780405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.798226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 269/528] 10.222.0.88 () {56 vars in 1139 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:36.799085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.905246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790d2a1-e291-46b4-880d-7ed615e5327e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.917969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc26c2-a2fe-42a3-9a4d-81c1f37761e2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:36.943232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.943899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.944184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.944464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.945249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:36.945564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.947181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.960067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 260/529] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:36.960729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:36.961401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:36.961687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:36.961991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:36.962712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:36.963018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:36.964657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:36.965883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.969503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:36.981299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 270/530] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:36.981299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:36.998383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.084054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300e8bf-7615-488c-8e82-a4fae4f27488 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.087696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f9ef6-313e-4456-8249-57cee3d48920 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.120559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.121217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.121505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.121819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.122644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=ee056b132bd24851bd0e2afe9cb7170e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:37.122960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.124034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.124424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.124623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.124902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.125176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.138107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 261/531] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/role_assignments?user.id=ee056b132bd24851bd0e2afe9cb7170e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:37.138107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.215699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.225655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.227733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.253270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.258415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86312a9-2e14-4d37-8bd6-4ec00f3d7516 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.310974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.312046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.314105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.314105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.314445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=984fa8bcb66146e5ae376a16d1d1b3cf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:37.314975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.318168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.326778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 262/532] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/role_assignments?user.id=984fa8bcb66146e5ae376a16d1d1b3cf&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:37.327800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.375134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 271/533] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:37.381684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.514347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c1b74-c20d-4406-8ab3-0e65bafaf616 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.578340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.591454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.592276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.592561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.592838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.603334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.606315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.640764 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c6543-88d2-4522-9fb1-d7dd586fae21 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:37.642378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:37.643204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:37.644080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:37.644486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 263/534] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:37.649861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.677206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:37.717910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.719150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.719440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.719723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.742084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.743568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:37.777192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:37.778356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:37.779932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:37.779932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:37.779932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:37.779932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:37.780340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 272/535] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:37.794916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.826457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d824f-0658-4b5b-b830-5d6599fe1c54 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:37.881969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:37.883745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:37.884454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:37.885617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:37.886878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:37.889516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:37.891907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:37.908067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 264/536] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:37.915784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:37.983470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.000705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b03bc-7c11-4ad2-a5e0-e92ba94a61b5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.039045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d2c4-1dc1-4980-8056-a45686d869e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.040592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d2c4-1dc1-4980-8056-a45686d869e2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.041297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d2c4-1dc1-4980-8056-a45686d869e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.041875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d2c4-1dc1-4980-8056-a45686d869e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.042783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d2c4-1dc1-4980-8056-a45686d869e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:38.043264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d2c4-1dc1-4980-8056-a45686d869e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.043366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860dcba-a3de-4cfa-8897-1e8b6ec964af None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.045158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d2c4-1dc1-4980-8056-a45686d869e2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.067943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 273/537] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:38.067943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46d36b-4687-4bee-8b51-6bc8469c02ac None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.068335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46d36b-4687-4bee-8b51-6bc8469c02ac None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.068335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46d36b-4687-4bee-8b51-6bc8469c02ac None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.069841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 274/538] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:38.071119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46d36b-4687-4bee-8b51-6bc8469c02ac None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.092080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64de4b8-6a68-4d80-a45a-d968e7f53604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.093600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de4b8-6a68-4d80-a45a-d968e7f53604 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.094377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de4b8-6a68-4d80-a45a-d968e7f53604 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.094849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64de4b8-6a68-4d80-a45a-d968e7f53604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.096149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64de4b8-6a68-4d80-a45a-d968e7f53604 None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=982fcdf935f64b98aae10f206a21da2e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:38.096950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64de4b8-6a68-4d80-a45a-d968e7f53604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.098708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64de4b8-6a68-4d80-a45a-d968e7f53604 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.107032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.109342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 265/539] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:37 2022] GET /identity/v3/role_assignments?user.id=982fcdf935f64b98aae10f206a21da2e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:38.113332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa1f1-99fa-46ec-b533-f0f38559ecb6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.114068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa1f1-99fa-46ec-b533-f0f38559ecb6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.114497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa1f1-99fa-46ec-b533-f0f38559ecb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.141102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-245fa1f1-99fa-46ec-b533-f0f38559ecb6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:38.143087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-245fa1f1-99fa-46ec-b533-f0f38559ecb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:38.225752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 266/540] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:38.225973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46d36b-4687-4bee-8b51-6bc8469c02ac None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.249133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa1f1-99fa-46ec-b533-f0f38559ecb6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.309385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.312377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.313333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.313623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.313932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.330948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.332490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:38.359919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:38.360829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:38.361661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:38.367046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 275/541] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:38.368291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.379611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa1f1-99fa-46ec-b533-f0f38559ecb6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.427218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.428569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.429268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.430134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.443337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:38.452041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:38.452924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:38.453697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:38.458424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 267/542] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:38.464589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.545452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.584438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab43a5-1df2-47e0-88ea-76b9b97e8983 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.595492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fa6c9-5ecb-451b-abcb-bb076ba9e1a4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.605560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:38.634127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.634632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.634921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.635204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.635999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:38.636318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.637791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.644546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.645567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.646101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.646605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.647541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:38.648059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.649627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.661843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 276/543] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:38.662716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 268/544] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:38.667039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.668499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.791017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f14129-f3e9-42e9-8a00-b71b1c73f31e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.832516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.833826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.834325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.834850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.847773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.849563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:38.869635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:38.870514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:38.871368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:38.871368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:38.871368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:38.871368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:38.873729 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 269/545] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:38.884144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528fc7f-529e-406e-8623-b1de46d01293 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:38.891730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:38.925967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:38.925967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:38.925967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:38.925967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:38.926382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=33e61ed31e0943e8baf9a85412860245) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:38.926382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:38.926382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:38.938083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 277/546] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/role_assignments?user.id=33e61ed31e0943e8baf9a85412860245&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:38.948118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:39.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b577-f3ea-4f5b-9967-8b648d6eb758 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:39.125396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:39.126026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.126309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.126588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.127327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:39.127638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:39.139960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:39.154207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 270/547] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:39.161829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:39.206151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:39.294039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d5f48-3dc6-4363-a8e2-00f3c6fb2a2e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:39.331321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f24fbdc-26b4-4b39-8681-5754a77889dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:39.332671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24fbdc-26b4-4b39-8681-5754a77889dd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.332950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24fbdc-26b4-4b39-8681-5754a77889dd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.333338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24fbdc-26b4-4b39-8681-5754a77889dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.341643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24fbdc-26b4-4b39-8681-5754a77889dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:39.343190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f24fbdc-26b4-4b39-8681-5754a77889dd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4f24fbdc-26b4-4b39-8681-5754a77889dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:39.365152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:39.366143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:39.368183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:39.370507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 278/548] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:39.376483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:39.501736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:39.526863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1a684-3041-4880-8016-44556737d782 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:39.582036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d0d77-577b-4219-8ca2-ab6eb3ed191c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:39.583120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0d77-577b-4219-8ca2-ab6eb3ed191c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.583611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0d77-577b-4219-8ca2-ab6eb3ed191c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.584091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0d77-577b-4219-8ca2-ab6eb3ed191c None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.585159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d0d77-577b-4219-8ca2-ab6eb3ed191c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:39.585675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d0d77-577b-4219-8ca2-ab6eb3ed191c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:39.587361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d0d77-577b-4219-8ca2-ab6eb3ed191c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:39.612939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 271/549] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 15:59:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:39.614755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff65d83-f390-40bb-87d2-ad944ddb339d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.615055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff65d83-f390-40bb-87d2-ad944ddb339d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.615343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff65d83-f390-40bb-87d2-ad944ddb339d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.617683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 279/550] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:39.621817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b736d-a7ed-4bab-ae86-fb66bddd82eb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.622342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b736d-a7ed-4bab-ae86-fb66bddd82eb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.622827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b736d-a7ed-4bab-ae86-fb66bddd82eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.623115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff65d83-f390-40bb-87d2-ad944ddb339d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:39.647927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-388b736d-a7ed-4bab-ae86-fb66bddd82eb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:39.649465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-388b736d-a7ed-4bab-ae86-fb66bddd82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:39.710589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:39.747600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff65d83-f390-40bb-87d2-ad944ddb339d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:39.790367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ae0c5-4674-4f6b-9cf0-f9d92605b5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:39.791057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ae0c5-4674-4f6b-9cf0-f9d92605b5ca None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:39.791353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ae0c5-4674-4f6b-9cf0-f9d92605b5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:39.791647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ae0c5-4674-4f6b-9cf0-f9d92605b5ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:39.792268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 272/551] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:39.822812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b736d-a7ed-4bab-ae86-fb66bddd82eb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:39.887267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ae0c5-4674-4f6b-9cf0-f9d92605b5ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:39.887267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ae0c5-4674-4f6b-9cf0-f9d92605b5ca None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:39.939641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:39.998269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b736d-a7ed-4bab-ae86-fb66bddd82eb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.004351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.046141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 280/552] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:40.047823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c7e9-7463-4197-8aad-0d781c2d6e21 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.048258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c7e9-7463-4197-8aad-0d781c2d6e21 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.048683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805c7e9-7463-4197-8aad-0d781c2d6e21 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.054487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e5eab-5b6e-4acf-86e6-d6c0031e734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.054487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e5eab-5b6e-4acf-86e6-d6c0031e734b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.054487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e5eab-5b6e-4acf-86e6-d6c0031e734b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.054487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e5eab-5b6e-4acf-86e6-d6c0031e734b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.055645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 281/553] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:40.056643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c7e9-7463-4197-8aad-0d781c2d6e21 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.068514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e5eab-5b6e-4acf-86e6-d6c0031e734b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5a0e5eab-5b6e-4acf-86e6-d6c0031e734b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:40.078898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:40.080016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:40.080837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:40.081473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 273/554] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:40.083450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82cc0f-69d9-47e0-9c26-2988367ba8ee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.084607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82cc0f-69d9-47e0-9c26-2988367ba8ee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.085066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82cc0f-69d9-47e0-9c26-2988367ba8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.110338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4d82cc0f-69d9-47e0-9c26-2988367ba8ee None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:40.111938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d82cc0f-69d9-47e0-9c26-2988367ba8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:40.180870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805c7e9-7463-4197-8aad-0d781c2d6e21 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.213595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 274/555] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:40.215071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82cc0f-69d9-47e0-9c26-2988367ba8ee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.224771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.225617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.225943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.226223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.240239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.241866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:40.266484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:40.267522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:40.273554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:40.273554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:40.273554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:40.273554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:40.273554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 282/556] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:40.273554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.301865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.302623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.359124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82cc0f-69d9-47e0-9c26-2988367ba8ee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.405607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.406614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.407076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.407449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641f063b-bd67-42d6-b46b-1007a5233b03 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.407736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.408827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:40.410230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.412759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.425138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 275/557] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:40.426981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.449739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.450907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.451291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.451675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.466066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:40.477756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:40.479258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:40.480083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:40.484838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 283/558] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:40.485976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.571843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0ca0d-4801-48d8-9ca9-37876e8abd42 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.600477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.611099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.620604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.622599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.623056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.623481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.624797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:40.625740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.627843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.632825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee919aa1-7049-4b8d-9dfe-de8472e13554 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.645330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 276/559] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:40.646511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.685488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.686756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.687143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.687793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.702077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.704353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:40.740986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:40.742796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:40.744268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:40.744268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:40.744268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:40.744268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:40.744268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 284/560] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:40.744605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.800271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5195162-67c3-4670-9aba-01cb38439255 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.838357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.839041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.839581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.839859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.840605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:40.840910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.842317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.855701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 277/561] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:40.856489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.872093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b48b3-e21d-4895-aa74-539cb158a162 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:40.905947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.915627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601fac4-f6cd-4e1b-8db6-f07b7db79cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:40.916986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fac4-f6cd-4e1b-8db6-f07b7db79cce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.917313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fac4-f6cd-4e1b-8db6-f07b7db79cce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.917587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601fac4-f6cd-4e1b-8db6-f07b7db79cce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.921089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:40.927246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601fac4-f6cd-4e1b-8db6-f07b7db79cce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:40.928982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d601fac4-f6cd-4e1b-8db6-f07b7db79cce None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d601fac4-f6cd-4e1b-8db6-f07b7db79cce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:40.944398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:40.945488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 285/562] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aba17f-6ff3-4e1d-8da7-967da67e86a8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:40.947162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aba17f-6ff3-4e1d-8da7-967da67e86a8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:40.947835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aba17f-6ff3-4e1d-8da7-967da67e86a8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:40.952768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 286/563] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:40.953576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aba17f-6ff3-4e1d-8da7-967da67e86a8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:40.978337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01921e-6d41-4aed-bfb8-53a13dbfcb24 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.016517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.017202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.017489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.017804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.018534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.018846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.020268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.032838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 278/564] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.033669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.078577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aba17f-6ff3-4e1d-8da7-967da67e86a8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.119441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cc242-7d4d-44d1-a145-e28a530c7d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.120246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cc242-7d4d-44d1-a145-e28a530c7d4a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.120541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cc242-7d4d-44d1-a145-e28a530c7d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.120819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cc242-7d4d-44d1-a145-e28a530c7d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.132160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020cc242-7d4d-44d1-a145-e28a530c7d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.135644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020cc242-7d4d-44d1-a145-e28a530c7d4a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-020cc242-7d4d-44d1-a145-e28a530c7d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:41.153510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:41.154513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:41.155403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:41.155403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:41.155403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:41.155403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:41.155403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 287/565] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:41.156014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03dbad-9f57-432e-b99f-4cad7eaf3414 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.156270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03dbad-9f57-432e-b99f-4cad7eaf3414 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.156553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03dbad-9f57-432e-b99f-4cad7eaf3414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.166115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c704b6-61b8-4582-b61f-7d202d17f023 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.168417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.174023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3b03dbad-9f57-432e-b99f-4cad7eaf3414 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:41.175258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3b03dbad-9f57-432e-b99f-4cad7eaf3414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:41.205699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.206401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.206680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.206957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.207774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.208047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.209608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.223048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 279/566] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.223763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.230117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.269895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 288/567] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:41.269895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03dbad-9f57-432e-b99f-4cad7eaf3414 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.341265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84225cb1-dca7-488b-9aaf-09e128b5c0ee None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.380668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.381472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.381776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.382076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.386396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03dbad-9f57-432e-b99f-4cad7eaf3414 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.390042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.391524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:41.406278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:41.407281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:41.408225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 280/568] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:41.408842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.432354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.433353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.433641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.433952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.434688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.435000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.436757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.437865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.456260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 289/569] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.456260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.513625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.524009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2cb266-905d-4f7e-8305-aebecb131d9f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.570483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.571182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.571462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.571750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.572589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=9b4b1f0775ba4875a550ea986ed483e7) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.572907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.574385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.580102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 281/570] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/role_assignments?user.id=9b4b1f0775ba4875a550ea986ed483e7&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.580884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.588696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817a9db-849c-4ae8-b8f2-d198d664943c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.629812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.630631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.631032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.631341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.643847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:41.651894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:41.652955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:41.653853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 290/571] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:41.655718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.694350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.712329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520681c-969a-4f6a-bac7-01535dd4a8d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.756055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f2b3d-1972-4d3e-8931-1845bcc61383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.756757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f2b3d-1972-4d3e-8931-1845bcc61383 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.757050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f2b3d-1972-4d3e-8931-1845bcc61383 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.757338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f2b3d-1972-4d3e-8931-1845bcc61383 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.758405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f2b3d-1972-4d3e-8931-1845bcc61383 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.758904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f2b3d-1972-4d3e-8931-1845bcc61383 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.760674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f2b3d-1972-4d3e-8931-1845bcc61383 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.769229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:41.775569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2f96a-71a5-438c-b651-29a929a54a1c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.781016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 282/572] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.782482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454d433-afc0-4538-99aa-a2c54c820680 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.782778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454d433-afc0-4538-99aa-a2c54c820680 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.783062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454d433-afc0-4538-99aa-a2c54c820680 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.784350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 283/573] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:41.785592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454d433-afc0-4538-99aa-a2c54c820680 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.824362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.825228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.825513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.825825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.835163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.836633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:41.852444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:41.854244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:41.856450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:41.858786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 291/574] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:41.859299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:41.929968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454d433-afc0-4538-99aa-a2c54c820680 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.974584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ef9c-50ad-40ba-b372-92e717596dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:41.975329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ef9c-50ad-40ba-b372-92e717596dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.975613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ef9c-50ad-40ba-b372-92e717596dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.975889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ef9c-50ad-40ba-b372-92e717596dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:41.976644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ef9c-50ad-40ba-b372-92e717596dc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:41.977011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ef9c-50ad-40ba-b372-92e717596dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:41.978442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ef9c-50ad-40ba-b372-92e717596dc7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:41.979224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5ac0e-5bcf-4952-9be0-5d2a135ac34c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:41.990035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 284/575] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:41.991321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca8683-fac3-44aa-8a44-7f308e1a361c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:41.991658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca8683-fac3-44aa-8a44-7f308e1a361c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:41.991940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca8683-fac3-44aa-8a44-7f308e1a361c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.005540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.012521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4bca8683-fac3-44aa-8a44-7f308e1a361c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:42.013866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4bca8683-fac3-44aa-8a44-7f308e1a361c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:42.023009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.023810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.024093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.024367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.043860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.045337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:42.064673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.066439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.068141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:42.068141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:42.068141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.068141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:42.068141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 292/576] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:42.068141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.084259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.099135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 285/577] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:42.100282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca8683-fac3-44aa-8a44-7f308e1a361c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.224893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d2370-6dc0-4ccc-b622-fe1301b271da None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.229129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca8683-fac3-44aa-8a44-7f308e1a361c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.263634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.264309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.264594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.264868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.269085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:42.269085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.269085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.270287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.271104 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.271385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.271661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.275146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.284810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.286856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.294501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 293/578] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:42.294501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:42.306851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.307849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.308749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:42.308749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:42.308749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.308749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:42.308749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 286/579] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:42.309030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.375900 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.416664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0cecd-3970-4404-b636-02ca21a5bf56 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.427886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b8615f-beb5-480d-9f48-cc87a909e878 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.470677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.471719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.472189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.473216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.474399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:42.474915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.477276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.489848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.491041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.491511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.491993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.503363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 294/580] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:42.503363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.509105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:42.517023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.518016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:42.518930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 287/581] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:42.520146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.576144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.619417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130369c5-34d0-4505-b855-9fa79262c632 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.639593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f41a35-1fd5-47c9-a9d2-725ece2d77ad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.648631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.657342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.658187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.658559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.658761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.670494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.671948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.678076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.678759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.679076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.679330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.680055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:42.680366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.681766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:42.690217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.691680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.692567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:42.692567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:42.692567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:42.692567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:42.692567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 295/582] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:42.692864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.704248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 288/583] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:42.704248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.816032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10ca45-7a47-4e84-a8f1-0d59473769f5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.817463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b62c65-9de3-4f23-9009-e36ffc78d78d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:42.831547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.854492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.855340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.855634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.855921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.862883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:42.863591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:42.863880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:42.864161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:42.864566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.864943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:42.865252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:42.866088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.866688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:42.881750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 289/584] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:42.881750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:42.882179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:42.883041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 296/585] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:42.884476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:42.931385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:42.994150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55da7b-59d3-420e-a507-a015af7bd069 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.008788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa529bed-d06b-4421-be93-7b77abb05ccb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.042539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.043381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.043627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.043898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.045995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.046915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.047209 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.047485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.048199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.048505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.049902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.052802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.054276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.069304 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 290/586] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:43.069304 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:43.070118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:43.071326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:43.072266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 297/587] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:43.072866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.105446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.185676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.189123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340aa4f-7a79-4a32-9d68-75021ba94074 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.189543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2417d6-6fe0-4286-ab8c-b9728ce0367c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.228591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.229226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.229396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.229673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.229981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.230594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.230967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.231759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.232080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.233632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.243661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.244664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.252531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 298/588] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:43.252531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:43.265025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:43.266251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:43.267392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:43.267392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:43.267392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:43.267392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:43.271942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 291/589] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:43.272859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.366578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d4645-19ea-4048-87cc-2e19c5440466 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.367616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.388902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf81a66-6b5a-4be0-88ca-1912acddce96 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.405312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.406154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.406465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.406806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.417412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.419853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.428888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.429571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.429929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.430206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.431053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.431374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.432823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:43.440688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:43.441645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:43.442569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:43.446690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 299/590] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:43.447298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.459377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 292/591] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:43.459377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.475510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.562437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec987e-1197-49ab-85f5-644755e7d4c0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.570464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7a958-b57e-4592-8c72-10b8ae49fb41 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.601206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.602138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.602428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.602711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/ee056b132bd24851bd0e2afe9cb7170e/roles/763f046b87da4abbaff504f77c55288c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.610906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.612064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.612353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.612634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.613562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.613903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.615400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.618430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=ee056b132bd24851bd0e2afe9cb7170e, role_id=763f046b87da4abbaff504f77c55288c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.619814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.624332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.629294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 293/592] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:43.630315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.639691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 300/593] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:43 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/ee056b132bd24851bd0e2afe9cb7170e/roles/763f046b87da4abbaff504f77c55288c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:43.640986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.733573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.764854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c614bec-b545-4169-b953-24a2a0a621b3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.783637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd033d0-3b28-44cd-b572-446957cb5eae None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:43.807410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.808455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.809032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.809309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.810151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:43.810462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.811960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.829902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 294/594] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:43.829902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.830655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9e27f-04ed-4a5e-ab0a-3a0fd4033a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:43.831842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e27f-04ed-4a5e-ab0a-3a0fd4033a17 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.832125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e27f-04ed-4a5e-ab0a-3a0fd4033a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.832404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e27f-04ed-4a5e-ab0a-3a0fd4033a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.847959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9e27f-04ed-4a5e-ab0a-3a0fd4033a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:43.849851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9e27f-04ed-4a5e-ab0a-3a0fd4033a17 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-93d9e27f-04ed-4a5e-ab0a-3a0fd4033a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:43.874076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:43.874999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:43.875940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:43.875940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:43.875940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:43.875940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:43.875940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 301/595] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:43.877295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ee4f3-6adc-415b-980f-36360b5d9249 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:43.877295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ee4f3-6adc-415b-980f-36360b5d9249 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:43.877469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ee4f3-6adc-415b-980f-36360b5d9249 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:43.879040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 302/596] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:43.881859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ee4f3-6adc-415b-980f-36360b5d9249 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:43.912399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:43.960411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b8dce-76e5-4183-bb93-f6047a750025 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.000953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dfe92-b405-4499-8388-affe30355cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.001196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ee4f3-6adc-415b-980f-36360b5d9249 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.001791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dfe92-b405-4499-8388-affe30355cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.002084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dfe92-b405-4499-8388-affe30355cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.002367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dfe92-b405-4499-8388-affe30355cd0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.012971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dfe92-b405-4499-8388-affe30355cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.015181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dfe92-b405-4499-8388-affe30355cd0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.028735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-eb0dfe92-b405-4499-8388-affe30355cd0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:44.031233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:44.032213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:59:44.033201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:44.037668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 295/597] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:44.038972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff192-6286-4be4-977d-3827341691fb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.039523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff192-6286-4be4-977d-3827341691fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.039523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ff192-6286-4be4-977d-3827341691fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.044874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.045792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.046136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.046440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/984fa8bcb66146e5ae376a16d1d1b3cf/roles/763f046b87da4abbaff504f77c55288c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.057095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4f0ff192-6286-4be4-977d-3827341691fb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:44.058340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f0ff192-6286-4be4-977d-3827341691fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:44.064001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=984fa8bcb66146e5ae376a16d1d1b3cf, role_id=763f046b87da4abbaff504f77c55288c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.065618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.089802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 303/598] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:43 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/984fa8bcb66146e5ae376a16d1d1b3cf/roles/763f046b87da4abbaff504f77c55288c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:44.089802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.156987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 296/599] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:44.158256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff192-6286-4be4-977d-3827341691fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.231321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.246489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f31990-8534-41aa-8662-00ffde79c1d9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.293768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.294454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.294741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.295023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.297125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:44.297125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.299250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.313225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ff192-6286-4be4-977d-3827341691fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.313588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 304/600] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:44.314826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.355021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.355363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.355856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.356226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.356504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.357354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:44.357664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.360455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.405630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 297/601] 10.222.0.88 () {56 vars in 1139 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:44.408011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.451446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb54fb3-57be-4c24-ba80-34c34fe8c54c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.492064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.492883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.493516 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.493639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.506799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:44.515499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:44.516468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:44.517280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 305/602] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:44.523336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.569184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9812e-d3c6-44a7-a3a8-46c4e07265e2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.579605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.608774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.610079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.610583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.611064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.619515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.620277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.622292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.631504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b30fe-461a-4474-b1e9-b6e51b841314 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:44.638274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:44.639163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:44.640052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:44.642767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 298/603] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:44.643787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.679795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.680799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.681089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.681375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.695785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.697828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:44.718096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:44.719105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:44.720144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:44.720144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:44.720144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:44.720144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:44.722673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 306/604] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:44.723736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.767748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddb205-bbfd-4926-8aea-c9b81f06b93d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.805209 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.805857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.806150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.806437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.807334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:44.807707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.809264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.823787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 299/605] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:44.823787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.846231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c18c7f-42e4-49a6-8719-ae3d679a8a99 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.857931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.886668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.887720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.888741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.889860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:44.890512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.892193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:44.909005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 307/606] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:44.910985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:44.919827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:44.954808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadd18e-4953-4655-9cd3-3916459ff87a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:44.995040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:44.995728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:44.996018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:44.996303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:44.997095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:44.997415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:44.998983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.017028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 300/607] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:45.018051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.051864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7ade2-c2f4-4298-a7e0-e636e157ddeb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.092665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.093643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.093983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.094537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.105984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.107717 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:45.127503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:45.128445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:45.129998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:45.129998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:45.129998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:45.129998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:45.134759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.135059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 308/608] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:45.137764 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.170423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cbb5c-d030-47bc-a0f8-04071b9d575a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.202790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.215083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.215906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.216694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.216812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/982fcdf935f64b98aae10f206a21da2e/roles/763f046b87da4abbaff504f77c55288c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.233166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=982fcdf935f64b98aae10f206a21da2e, role_id=763f046b87da4abbaff504f77c55288c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.234838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.257829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 301/609] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:45 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/982fcdf935f64b98aae10f206a21da2e/roles/763f046b87da4abbaff504f77c55288c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:45.259196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.262989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219b862-0a35-4dcd-a9c0-6f0e60949c2e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.351549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.355130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.358179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.358959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.383592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.386849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:45.422843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:45.424052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:45.425356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 309/610] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:45.428443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.534603 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa99e5-1c55-470e-bc24-05c151268fab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.557100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc3ebe-4e2b-4141-85f6-d6e2a7c94235 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.576191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.576870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.577274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.577958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.578377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:45.578692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.580086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.581104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.594990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.605764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 302/611] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:45.607362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.610417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.613237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.613774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.615307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.616287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:45.620291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.622010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.638291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 310/612] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:45.644181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.764537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139fac8-37c7-4bbf-8fef-ab797f0aefdb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.815018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.816810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.817246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.817673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/33e61ed31e0943e8baf9a85412860245/roles/763f046b87da4abbaff504f77c55288c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.844082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5176a-3c73-4a50-ac45-6957eef75af4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:45.852024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=33e61ed31e0943e8baf9a85412860245, role_id=763f046b87da4abbaff504f77c55288c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.854198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.880995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 311/613] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:45 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/33e61ed31e0943e8baf9a85412860245/roles/763f046b87da4abbaff504f77c55288c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:45.890380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524206b-c89f-40f3-bffc-e6721bd61733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:45.891211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524206b-c89f-40f3-bffc-e6721bd61733 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.891497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524206b-c89f-40f3-bffc-e6721bd61733 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.891775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524206b-c89f-40f3-bffc-e6721bd61733 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.905306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524206b-c89f-40f3-bffc-e6721bd61733 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:45.915102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524206b-c89f-40f3-bffc-e6721bd61733 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a524206b-c89f-40f3-bffc-e6721bd61733 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:45.939744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:45.940987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:45.942236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 303/614] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:45.947253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:45.996484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891424e-0b2e-42c6-8859-3b97df95ebe0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:45.996722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891424e-0b2e-42c6-8859-3b97df95ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:45.996984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891424e-0b2e-42c6-8859-3b97df95ebe0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:45.998664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:45.998799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 304/615] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:46.006234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4070-85c1-470e-96bc-353e58d57cc9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.006562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4070-85c1-470e-96bc-353e58d57cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.006803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a4070-85c1-470e-96bc-353e58d57cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.044833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4f6a4070-85c1-470e-96bc-353e58d57cc9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:46.049204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f6a4070-85c1-470e-96bc-353e58d57cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:46.140453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350386d-ef76-4e03-882d-d2909108980a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.151669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.166611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 305/616] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:46.200678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4070-85c1-470e-96bc-353e58d57cc9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.201666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.202551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.202989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.203404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.204419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:46.204870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.206698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.234542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 312/617] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 15:59:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:46.241050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.358221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.385991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240450-26bb-46d8-8d25-a5e285104e8b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.388624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a4070-85c1-470e-96bc-353e58d57cc9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.440772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.442060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.442354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.442631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.454286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:46.462978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:46.464021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:46.465131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 306/618] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:46.471037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.499170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.500231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.500518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.500795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.517250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.521359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:46.539493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:46.540804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:46.541925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:46.541925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:46.541925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:46.541925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:46.543310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 313/619] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:46.548901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.560754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.620539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b375d-4f9c-47aa-9ae0-6b2eb81b05c8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.660459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d678f-53f0-42f0-a97e-58d3943a2a16 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.665557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc61802-307f-4621-a64c-d76655212cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.666414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61802-307f-4621-a64c-d76655212cfc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.666811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61802-307f-4621-a64c-d76655212cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.667204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc61802-307f-4621-a64c-d76655212cfc None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.668481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61802-307f-4621-a64c-d76655212cfc None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:46.668915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61802-307f-4621-a64c-d76655212cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.670800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc61802-307f-4621-a64c-d76655212cfc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.682808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 307/620] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:46.684361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37869b3-2f15-4253-9cc9-22a391b3e55b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.684767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37869b3-2f15-4253-9cc9-22a391b3e55b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.685160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37869b3-2f15-4253-9cc9-22a391b3e55b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.686969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 308/621] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:46.693292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95e116-4012-437b-b38e-dab1a4fbb484 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.694183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95e116-4012-437b-b38e-dab1a4fbb484 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.694632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95e116-4012-437b-b38e-dab1a4fbb484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.716009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0f95e116-4012-437b-b38e-dab1a4fbb484 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:46.717095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.717678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f95e116-4012-437b-b38e-dab1a4fbb484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:46.718397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.718946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.719536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.720903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:46.721527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.724763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.729423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.748022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 314/622] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:46.749650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.800366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 309/623] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:46.801884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95e116-4012-437b-b38e-dab1a4fbb484 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:46.847140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:46.909413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95e116-4012-437b-b38e-dab1a4fbb484 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.943765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67e63bb-ab9c-4869-9896-e83cb9b26aef None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:46.945254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:46.946095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:46.946383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:46.946664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:46.955438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:46.957825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:46.972979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:46.973969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:46.974855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:46.978905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 310/624] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:46.979751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.007057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.008397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.009030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.009533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.031838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.034743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:47.071156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:47.073345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:47.075519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:47.075519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:47.075519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:47.075519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:47.080606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 315/625] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:47.082172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.085566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a28e101-a32b-4f5e-8273-c825704563dd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.087257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.122683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.123137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.123426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.123712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.135375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:47.143276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:47.145323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 311/626] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:47.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.183585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.255307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a88755-7aa1-4e8b-91c3-f9445710c5b5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.289172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db486ed-92b0-496d-b534-801ba444c3da None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.291329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.292004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.292299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.292581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.293305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:47.293611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.295056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.313753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 312/627] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:47.313753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.335779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.370474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.372243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.372966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.373865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.375755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:47.376541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.380255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.404551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 316/628] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:47.406331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.423369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a0b9d-221e-403b-b8bd-1abfbde38ad0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.459691 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.460516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.460811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.461098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.461894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:47.462212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.463903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.473255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 313/629] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:47.474121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.538039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d945d-6749-47ea-aba2-3176f7f38e35 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.549142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.563538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.581219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903103b8-3133-4464-bc79-db1e0898a279 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.582633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.583817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.584395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.584894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.593658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.595462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:47.611882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:47.613035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:47.614893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:47.621420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.621420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.621420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.621420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/9b4b1f0775ba4875a550ea986ed483e7/roles/763f046b87da4abbaff504f77c55288c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.622162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 317/630] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:47.623195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.638922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=9b4b1f0775ba4875a550ea986ed483e7, role_id=763f046b87da4abbaff504f77c55288c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.640743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.674605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 314/631] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 15:59:47 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/9b4b1f0775ba4875a550ea986ed483e7/roles/763f046b87da4abbaff504f77c55288c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 15:59:47.676612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.811114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f9d83-f08b-449c-b035-34335a0c89c1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.843184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0dad6-17be-4efa-ae4f-2760e490b03f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:47.890716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8a90-310b-463b-bbd9-7c5a5323b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.891484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8a90-310b-463b-bbd9-7c5a5323b712 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.891888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8a90-310b-463b-bbd9-7c5a5323b712 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.892281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8a90-310b-463b-bbd9-7c5a5323b712 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.893252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8a90-310b-463b-bbd9-7c5a5323b712 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:47.893575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8a90-310b-463b-bbd9-7c5a5323b712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.893990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893d35e-801c-4d5e-82fc-3b5c4d48dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:47.894968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d35e-801c-4d5e-82fc-3b5c4d48dfbd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.895173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8a90-310b-463b-bbd9-7c5a5323b712 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.895483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d35e-801c-4d5e-82fc-3b5c4d48dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.895866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893d35e-801c-4d5e-82fc-3b5c4d48dfbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.897351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:47.915549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893d35e-801c-4d5e-82fc-3b5c4d48dfbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:47.917022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893d35e-801c-4d5e-82fc-3b5c4d48dfbd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:47.923178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 315/632] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:47.924671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19435d4-6dbb-4548-8dc5-2aa50091a294 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.924966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19435d4-6dbb-4548-8dc5-2aa50091a294 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.925250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19435d4-6dbb-4548-8dc5-2aa50091a294 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.927081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 316/633] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:47.930969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19435d4-6dbb-4548-8dc5-2aa50091a294 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e893d35e-801c-4d5e-82fc-3b5c4d48dfbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:47.940057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:47.941010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:47.942636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:47.942636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:47.942636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:47.942636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:47.942636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 318/634] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:47.943364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf4e9-0e84-4cee-9ca7-f1596e4d7234 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:47.943656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf4e9-0e84-4cee-9ca7-f1596e4d7234 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:47.943936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefbf4e9-0e84-4cee-9ca7-f1596e4d7234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:47.969086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fefbf4e9-0e84-4cee-9ca7-f1596e4d7234 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:47.970823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fefbf4e9-0e84-4cee-9ca7-f1596e4d7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:47.993866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.067813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 319/635] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:48.073742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf4e9-0e84-4cee-9ca7-f1596e4d7234 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.107035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19435d4-6dbb-4548-8dc5-2aa50091a294 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.174364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.175789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.177431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.177947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.228747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefbf4e9-0e84-4cee-9ca7-f1596e4d7234 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.262180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.277864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.280053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.283143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.285134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.285600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.286092 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.287007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:48.287494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.289094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.308425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 320/636] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:48.309577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.331625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.430561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 317/637] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 15:59:48.431878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.463077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62fec4-a8f1-416c-a495-3fa15c8d2c38 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.500610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2af9a-b4d3-4342-9304-caa339497e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.501548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2af9a-b4d3-4342-9304-caa339497e9f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.501932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2af9a-b4d3-4342-9304-caa339497e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.502209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2af9a-b4d3-4342-9304-caa339497e9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.513169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2af9a-b4d3-4342-9304-caa339497e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1dd2af9a-b4d3-4342-9304-caa339497e9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:48.520913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:48.521913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:48.522749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 321/638] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:48.523916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001780-63de-43d6-beae-71e0e6a6d603 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.523916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001780-63de-43d6-beae-71e0e6a6d603 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.524162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de001780-63de-43d6-beae-71e0e6a6d603 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.525482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 322/639] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:48.530388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001780-63de-43d6-beae-71e0e6a6d603 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.542610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.555542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaaa8d9-8679-4202-b458-7bd51af7f67f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.591876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.592595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.592877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.593151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.601337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.602828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:48.619027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:48.620055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 15:59:48.620975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:48.624319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.624956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 318/640] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:48.625732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.667333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de001780-63de-43d6-beae-71e0e6a6d603 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.715139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa18116-c4f3-472d-9878-5bdc19ba9c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.716262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa18116-c4f3-472d-9878-5bdc19ba9c35 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.716547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa18116-c4f3-472d-9878-5bdc19ba9c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.716829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa18116-c4f3-472d-9878-5bdc19ba9c35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.729526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa18116-c4f3-472d-9878-5bdc19ba9c35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.731057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa18116-c4f3-472d-9878-5bdc19ba9c35 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.749115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af3506-60d1-4544-a387-89618328f0f7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-faa18116-c4f3-472d-9878-5bdc19ba9c35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:48.750038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:48.751140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:48.752076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:48.752076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:48.752076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:48.752076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:48.753599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 323/641] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:48.755113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9011c0-fb74-44ab-a100-791432a9a99d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.755535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9011c0-fb74-44ab-a100-791432a9a99d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.755960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9011c0-fb74-44ab-a100-791432a9a99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.773603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bc9011c0-fb74-44ab-a100-791432a9a99d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:48.775070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bc9011c0-fb74-44ab-a100-791432a9a99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:48.796529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.797256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.797681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.797987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.799205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:48.799520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.801868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:48.811313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 319/642] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:48.812419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.830790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.862067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 324/643] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:48.863267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9011c0-fb74-44ab-a100-791432a9a99d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:48.935764 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:48.938333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7211c1-7e9b-4dcc-9739-2e13f2e54342 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:48.976222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:48.976900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:48.977176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:48.977448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:48.978321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:48.978632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:48.980086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.001797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9011c0-fb74-44ab-a100-791432a9a99d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.004806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 320/644] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:49.005674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.038145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.039087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.039751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.040031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.041016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:49.041328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.042994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.058043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 325/645] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:49.059083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.091127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.133793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe1e0d-9fc2-4155-b6f2-c23f3d006ed0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.180687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.181773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.182149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.182457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.192848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbb517-877a-4778-b35a-3e52801273c0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.197745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.199876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.207053 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:49.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:49.223129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:49.223129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:49.223129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:49.223129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:49.223129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 321/646] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:49.224285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.236790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.237617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.237989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.238795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.249375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:49.258562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:49.259546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 326/647] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:49.260725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.349047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854c526-6030-4fc5-bb07-ae0a40b4cf93 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.374962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.388983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2d88e-b832-4cf3-81f2-ddc4aa3d470b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.391174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.391834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.392121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.392403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.429644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.430745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.431106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.431461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.439618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.441248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:49.457288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:49.458459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:49.459465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 327/648] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:49.464452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.467940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.470300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.499256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.583541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458629b7-2bd8-40f2-be3c-bdc972dbd1ab None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.585105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 322/649] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:49.587580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.618657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.623074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.623749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.624035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.624448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.625300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:49.625778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.627121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.641058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 328/650] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:49.642729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.701959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602ca43-04ec-4c90-8dbb-9e9cdb3a1c31 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.740552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.741230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.741509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.741814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.742559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:49.742866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.744233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.752980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.758300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 323/651] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:49.758300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.768817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f566f1f-b176-4bee-b748-9896aa1a5931 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.806948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.807904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.808432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.808844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.809924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:49.810237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.811780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.826167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 329/652] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:49.827015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.879814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:49.898695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c37c-9fbe-4f8c-9f9d-9c5e2b8931eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.948121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d01481-1b14-400c-8d3c-b9ad2a37c2e9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:49.948359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.949169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.949459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.949742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:49.963022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:49.964653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:49.986083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:49.987070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:49.987959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:49.987959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:49.987959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:49.987959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:49.987959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 324/653] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:49.988357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:49.994303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:49.995122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:49.995406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:49.995684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.006565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.008127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:50.024362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:50.027689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 330/654] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:50.032425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.055887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.109180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9dd9c9-449a-4557-9352-c7b2e5457cad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.143802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3e1ce-3339-4cad-a1c6-2487b5ee640a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.147729 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.148528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.148862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.149088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.155972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.160807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.162301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.178345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d128e2-fbbe-4188-b6d9-1765c7c1746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.178997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d128e2-fbbe-4188-b6d9-1765c7c1746a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:50.179131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:50.180469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:50.182178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:50.182178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:50.182178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:50.182178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:50.182626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d128e2-fbbe-4188-b6d9-1765c7c1746a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.182626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d128e2-fbbe-4188-b6d9-1765c7c1746a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.182853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 325/655] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:50.182994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d128e2-fbbe-4188-b6d9-1765c7c1746a None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=ee056b132bd24851bd0e2afe9cb7170e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:50.182994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d128e2-fbbe-4188-b6d9-1765c7c1746a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.183246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.183382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d128e2-fbbe-4188-b6d9-1765c7c1746a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.191152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 331/656] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/role_assignments?user.id=ee056b132bd24851bd0e2afe9cb7170e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:50.192939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648205d-7263-498c-a076-8e187052ae54 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.193480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648205d-7263-498c-a076-8e187052ae54 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.193480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648205d-7263-498c-a076-8e187052ae54 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.194684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 332/657] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:50.196118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648205d-7263-498c-a076-8e187052ae54 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.298499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.326728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648205d-7263-498c-a076-8e187052ae54 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.330011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc5667-0916-46ed-901b-df789bdf37da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.370089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.371741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.372079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.372405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.373315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:50.373665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.374540 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf9071-aa77-40de-b52f-f803d80e6a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.375424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9071-aa77-40de-b52f-f803d80e6a34 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.375709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9071-aa77-40de-b52f-f803d80e6a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.375981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9071-aa77-40de-b52f-f803d80e6a34 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.377196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf9071-aa77-40de-b52f-f803d80e6a34 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:50.377685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf9071-aa77-40de-b52f-f803d80e6a34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.379322 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.379447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf9071-aa77-40de-b52f-f803d80e6a34 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.392512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 326/658] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:50.393352 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.402630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 333/659] 10.222.0.88 () {56 vars in 1137 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:50.403693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa5f41-934e-484a-b7d8-c56ccc520be7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.403993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa5f41-934e-484a-b7d8-c56ccc520be7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.404261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa5f41-934e-484a-b7d8-c56ccc520be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.422956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-09aa5f41-934e-484a-b7d8-c56ccc520be7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:50.424189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-09aa5f41-934e-484a-b7d8-c56ccc520be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:50.454977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.501428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 334/660] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:50.502786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa5f41-934e-484a-b7d8-c56ccc520be7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.512848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa304e9e-28de-455a-911b-3b240519d485 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.553629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.554399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.554682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.554959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.555680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:50.555989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.557365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.569571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 327/661] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:50.571757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.576783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.618623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa5f41-934e-484a-b7d8-c56ccc520be7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.654540 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.655349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.655824 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.656106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.663259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.664745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:50.679801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:50.680771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:50.681648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 335/662] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:50.682157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.696606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39315767-4fe6-4dcc-bf4b-1912f70f9236 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.711152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.740091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.740748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.741031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.741308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.803771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f352063-ca69-44a2-a0eb-bb1cb200bc9c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:50.822740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:50.824232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:50.828180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.840812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:50.841607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:50.841916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:50.842277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:50.854026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:50.861643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:50.862684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:50.863566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 336/663] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:50.864252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.948148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 328/664] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:50.949401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:50.966700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:50.996858 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37a20e-f345-44f4-8c60-ba32a05fc875 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.044484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.045395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.045677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.045984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.058283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.059742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.073016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559c89c-6d7a-43df-a02c-4daa9914d89a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:51.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:51.078007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 337/665] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:51.079372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.116690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.117356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.117632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.117934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.118715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.119029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.120464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.140348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.148232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 329/666] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:51.149243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.218289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf7581-3b7c-44de-8b14-7e6ef5ea5c0e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.235592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.256342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.257036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.257336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.257562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.258340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.258646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.260015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.274205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 338/667] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:51.274205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.278673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205b175-0b1f-41ae-ab04-629510057143 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.326023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.326825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.327106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.327800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.339545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.341294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:51.360896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:51.368509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:51.370357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:51.370357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:51.370357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:51.370357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:51.370357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 330/668] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:51.370357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.409871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0d9d6-0157-4f43-af86-0601ec25a7a1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.417325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.456297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.456975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.457816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.458095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.458944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.459255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.460692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.486093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 339/669] 10.222.0.88 () {56 vars in 1139 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:51.486865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.501339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e17164-95f5-4dec-8b89-e35765c6ed23 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.540656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.557901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.558800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.559171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.559537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.560729 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=984fa8bcb66146e5ae376a16d1d1b3cf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.561171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.563410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.577766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 331/670] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/role_assignments?user.id=984fa8bcb66146e5ae376a16d1d1b3cf&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:51.577766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.634975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618854a-2a43-452e-8305-a8c2be6a9572 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.687574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.687574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.687574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.687574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.698320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.700209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:51.726444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:51.727410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:51.728322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:51.728322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:51.728322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:51.728322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:51.728552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc234cbe-f857-4494-b2f6-bf68c153d4eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.728626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 340/671] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:51.729990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.757849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.773165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.773953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.774244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.774585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.776611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.776611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.777121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:51.800452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 332/672] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:51.801600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:51.873636 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:51.882159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a1f76-f164-4fbf-b623-0827ecb972b3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.946250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cc34a-e42e-4893-8c29-00920ff346e6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:51.969134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.971636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.972811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.973604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.987389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:51.988158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:51.989008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:51.989162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:51.989952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:51.990262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:51.991658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.005933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 333/673] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:52.006863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.122354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:52.123922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.152240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:52.175604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b365e64-c0d4-47dd-8e5a-ec5b373354b6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:52.186515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:52.219956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:52.220775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.221079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.221437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.232020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:52.233678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:52.253261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:52.255078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:52.256841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:52.266160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 334/674] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:52.268051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.273400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 341/675] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:52.279867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.391651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba356-bbb8-412c-94d5-0300e29d7ae5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:52.412760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:52.436939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:52.438289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.438855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.438966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ca7dc-83b4-4889-8282-f57c2262742a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:52.439503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.459340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:52.463369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.481859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f9b85-712a-4ab9-a54d-3d72bee1d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:52.482666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f9b85-712a-4ab9-a54d-3d72bee1d7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.482955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f9b85-712a-4ab9-a54d-3d72bee1d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.483234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f9b85-712a-4ab9-a54d-3d72bee1d7d5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.491238 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f9b85-712a-4ab9-a54d-3d72bee1d7d5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:52.492317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:52.493563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:52.494482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:52.494482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:52.494482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:52.494482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:52.494719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f9b85-712a-4ab9-a54d-3d72bee1d7d5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.497413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 342/676] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:52.498617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3c4f9b85-712a-4ab9-a54d-3d72bee1d7d5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:52.510025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:52.511035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:52.512105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 335/677] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:52.513376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d47daa-84e0-46a6-8a08-105ead23a755 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.513376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d47daa-84e0-46a6-8a08-105ead23a755 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.513376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d47daa-84e0-46a6-8a08-105ead23a755 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.517758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 336/678] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:52.517758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d47daa-84e0-46a6-8a08-105ead23a755 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.548528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:52.627720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d761-7c39-4c77-8953-26e45da723bc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:52.642982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d47daa-84e0-46a6-8a08-105ead23a755 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:52.665346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:52.666051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.666340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.666622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.667424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:52.667736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:52.669183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.681872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 343/679] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:52.682690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.688371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4e869-47a6-425a-a198-4ed3df6d8f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:52.689350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e869-47a6-425a-a198-4ed3df6d8f7d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.689574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e869-47a6-425a-a198-4ed3df6d8f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.689919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e869-47a6-425a-a198-4ed3df6d8f7d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.690842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4e869-47a6-425a-a198-4ed3df6d8f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:52.691224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4e869-47a6-425a-a198-4ed3df6d8f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:52.692849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4e869-47a6-425a-a198-4ed3df6d8f7d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.711074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 337/680] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:52.712567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d495f4c-45e7-4c53-a6a3-913066074bb0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.712859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d495f4c-45e7-4c53-a6a3-913066074bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.713134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d495f4c-45e7-4c53-a6a3-913066074bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.717397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:52.732898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4d495f4c-45e7-4c53-a6a3-913066074bb0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:52.734348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d495f4c-45e7-4c53-a6a3-913066074bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:52.807932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 338/681] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:52.808929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361d67b-fac4-47d3-960c-6381b28bcbdc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:52.810787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d495f4c-45e7-4c53-a6a3-913066074bb0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.854590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:52.855274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:52.855559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:52.855834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:52.856565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:52.856697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:52.857000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:52.858478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:52.881826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 344/682] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:52.881826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:52.977205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d495f4c-45e7-4c53-a6a3-913066074bb0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:52.993673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:53.011910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1849f8-49fd-4698-b573-c720cbcb778e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:53.038227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.039346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.039858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.040351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.041992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=982fcdf935f64b98aae10f206a21da2e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:53.042559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:53.045222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:53.053983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.054771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.055057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.055341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.066144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 339/683] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/role_assignments?user.id=982fcdf935f64b98aae10f206a21da2e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:53.066144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:53.075931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:53.077712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:53.096545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:53.098009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:53.099079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:53.103366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 345/684] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:53.104422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:53.175693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:53.211376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00e8f-7782-4b9a-8381-f4b982b8771c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:53.252861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.254239 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.254634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.255017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.267587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e680f3-e529-4845-8882-8c9224b0389a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:53.268412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:53.276825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:53.278148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 340/685] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:53.280190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:53.328608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.329747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.330280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.330672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.354382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:53.416579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fb360-7188-43c7-a9d3-bfe6fd8a1f15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:53.421314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:53.423032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:53.459705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.460728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.461258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.461822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.462955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:53.463489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:53.465510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:53.476712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:53.481073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 341/686] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:53.482404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:53.585392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 346/687] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:53.586814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:53.599081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2676fab-9f9c-4cbf-9a23-628ad5bfd335 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:53.637197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.638230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.638771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.639235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.640173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:53.640882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:53.642638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:53.646510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:53.656072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 342/688] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:53.657476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:53.717284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4145c4-6650-43f3-a81a-c5470c943183 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:53.728061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:53.760052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.761012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.761451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.761940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.763084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=33e61ed31e0943e8baf9a85412860245) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:53.763594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:53.765192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:53.773510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b6e4d-841b-45d5-9ada-19b71017a4b3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:53.778297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 347/689] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/role_assignments?user.id=33e61ed31e0943e8baf9a85412860245&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:53.779544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:53.825823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:53.826650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:53.826939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:53.827221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:53.848867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:53.850974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:53.871115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:53.872287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:53.873772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:53.873772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:53.873772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:53.873772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:53.876337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 343/690] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:53.886980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.011014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9b2dc-3778-4642-b01b-260ed8327488 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.049145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416010-e9cc-4f14-8cd2-d13a9ca9e7c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.069945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.083390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ebee9-792d-4bb4-975f-8c844aa58076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.084206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ebee9-792d-4bb4-975f-8c844aa58076 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.084493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ebee9-792d-4bb4-975f-8c844aa58076 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.084777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ebee9-792d-4bb4-975f-8c844aa58076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.101097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ebee9-792d-4bb4-975f-8c844aa58076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.104390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ebee9-792d-4bb4-975f-8c844aa58076 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.113514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d7d30-b3b0-448a-b30d-32467f45ea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.114551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7d30-b3b0-448a-b30d-32467f45ea31 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.115309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7d30-b3b0-448a-b30d-32467f45ea31 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.115771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d7d30-b3b0-448a-b30d-32467f45ea31 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.116683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7d30-b3b0-448a-b30d-32467f45ea31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:54.117165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7d30-b3b0-448a-b30d-32467f45ea31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.118763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d7d30-b3b0-448a-b30d-32467f45ea31 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e55ebee9-792d-4bb4-975f-8c844aa58076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:54.125892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:54.127136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:54.128094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:54.128094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:54.128094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:54.128094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:54.128811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 348/691] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:54.133752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff5866-2e99-48ae-9f9f-41d8c486ee4f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.133752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff5866-2e99-48ae-9f9f-41d8c486ee4f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.133752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff5866-2e99-48ae-9f9f-41d8c486ee4f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.133752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 349/692] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:54.144754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff5866-2e99-48ae-9f9f-41d8c486ee4f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.155786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 344/693] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:54.157115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8421c99-0467-485f-9c3b-0b0d497c1609 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.161756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8421c99-0467-485f-9c3b-0b0d497c1609 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.161756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8421c99-0467-485f-9c3b-0b0d497c1609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.181904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a8421c99-0467-485f-9c3b-0b0d497c1609 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:54.184363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a8421c99-0467-485f-9c3b-0b0d497c1609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:54.282410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 345/694] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:54.283724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8421c99-0467-485f-9c3b-0b0d497c1609 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.294868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff5866-2e99-48ae-9f9f-41d8c486ee4f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.338168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.339325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.339814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.340576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.341673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:54.342208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.344247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.358963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 350/695] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:54.365072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.401452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.422614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8421c99-0467-485f-9c3b-0b0d497c1609 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.427105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.474807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.475792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.476148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.476501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.487110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.489236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.515435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ccb67-929c-4003-a001-500611d4463f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:54.516495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:54.518031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:54.519056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:54.523713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 346/696] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:54.524643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.573518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.575341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.575881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.576426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.578311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.581484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:54.602961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:54.604088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 15:59:54.604935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:54.609883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 351/697] 10.222.0.88 () {56 vars in 1129 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:54.609883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.661313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23727b9-ce9e-49ac-b9bf-057ddf56f383 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.706315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386da424-9985-4019-8825-ba65357d7171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.707590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386da424-9985-4019-8825-ba65357d7171 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.708118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386da424-9985-4019-8825-ba65357d7171 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.708626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386da424-9985-4019-8825-ba65357d7171 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.718277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386da424-9985-4019-8825-ba65357d7171 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.720228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386da424-9985-4019-8825-ba65357d7171 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.741582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-386da424-9985-4019-8825-ba65357d7171 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:54.742367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:54.743320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 15:59:54.744392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:54.751922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 347/698] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:54.753222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386da424-9985-4019-8825-ba65357d7171 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.766733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a50d73-8d58-4be0-ae00-01e569590661 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.794424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:54.806937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.808038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.808874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.809393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.811446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:54.812054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.813919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.837622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 352/699] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:54.839234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.920390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386da424-9985-4019-8825-ba65357d7171 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:54.960805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:54.961901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:54.962401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:54.962930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:54.963855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:54.964438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:54.966392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:54.972000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 348/700] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:54.973224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:54.986635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1970788-ec78-41c4-8db2-62c5372a98c8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:55.029387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:55.029937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:55.030240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:55.030539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:55.031486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:55.031877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:55.033658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:55.060361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 353/701] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:55.060361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:55.060884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:55.074656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:55.088900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0638659-433e-4bca-9f55-707183d2fde0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:55.125210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:55.126091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:55.126479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:55.126909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:55.127793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=9b4b1f0775ba4875a550ea986ed483e7) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:55.128229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:55.129959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:55.142100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 349/702] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:54 2022] GET /identity/v3/role_assignments?user.id=9b4b1f0775ba4875a550ea986ed483e7&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:55.143226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:55.186919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d44eb1-5fae-4883-a1e3-2d8d93769b54 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:55.233980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:55.234813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:55.235202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:55.235572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:55.236407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:55.236821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:55.239157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:55.247690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 354/703] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 15:59:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:55.283083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae847f8-5c81-43a4-8701-7c0ae94b5488 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:55.326940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:55.327629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:55.327915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:55.328222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:55.355517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:55.413754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:55.415300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:55.534756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 350/704] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 15:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 15:59:55.541090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:55.620896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:55.665331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f593d7-98dd-42e1-8ab3-3411a21e56ec None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:55.702950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:55.703763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:55.704048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:55.704322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:55.717398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:55.719026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:55.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:55.739079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:55.739952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:55.739952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:55.739952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:55.739952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:55.739952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 355/705] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:55.745748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:55.819906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:55.888702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016287e0-091f-4448-a30f-1c91c08d24d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:55.927940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacdf8d-792a-4394-ba09-1a3ff701bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:55.928626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacdf8d-792a-4394-ba09-1a3ff701bff8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:55.928917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacdf8d-792a-4394-ba09-1a3ff701bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:55.929199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacdf8d-792a-4394-ba09-1a3ff701bff8 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:55.929963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacdf8d-792a-4394-ba09-1a3ff701bff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:55.930275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacdf8d-792a-4394-ba09-1a3ff701bff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:55.931672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacdf8d-792a-4394-ba09-1a3ff701bff8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:55.945064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 351/706] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:55.950990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:56.082658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f2a03-9a3f-40f7-9ced-56fc58541c9c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:56.111473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:56.131057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677efb7-b324-4539-9aeb-17b26ebdb548 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.131597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677efb7-b324-4539-9aeb-17b26ebdb548 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.131676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2677efb7-b324-4539-9aeb-17b26ebdb548 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.133030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 352/707] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:56.143710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ce014-db46-482e-b727-9c2c31b735d8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.144206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ce014-db46-482e-b727-9c2c31b735d8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.144667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ce014-db46-482e-b727-9c2c31b735d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.155768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:56.158125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.165642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.166221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.178125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:56.179883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:56.182806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4b7ce014-db46-482e-b727-9c2c31b735d8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:56.184017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b7ce014-db46-482e-b727-9c2c31b735d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 15:59:56.198465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:56.199416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 15:59:56.200369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:56.202622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 356/708] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:56.209687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:56.323119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 353/709] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 15:59:56.390193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ce014-db46-482e-b727-9c2c31b735d8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:56.409782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c577333-893d-4fe9-8e2e-8ef08f65e561 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:56.452048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:56.480946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261fd68-3544-4e18-92f1-c992e2e26bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:56.482155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261fd68-3544-4e18-92f1-c992e2e26bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.483267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261fd68-3544-4e18-92f1-c992e2e26bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.483940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261fd68-3544-4e18-92f1-c992e2e26bd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.486252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261fd68-3544-4e18-92f1-c992e2e26bd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:56.487005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261fd68-3544-4e18-92f1-c992e2e26bd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:56.490130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261fd68-3544-4e18-92f1-c992e2e26bd1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:56.513633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 357/710] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 15:59:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:56.514904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f584a3-0c5e-4e27-b63c-4fbd5b30e3f6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.515201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f584a3-0c5e-4e27-b63c-4fbd5b30e3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.515488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f584a3-0c5e-4e27-b63c-4fbd5b30e3f6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.521323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 358/711] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:56.522245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f584a3-0c5e-4e27-b63c-4fbd5b30e3f6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:56.549334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ce014-db46-482e-b727-9c2c31b735d8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:56.574809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:56.591887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30deed7c-532f-449c-b840-8417f4b806ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:56.593266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deed7c-532f-449c-b840-8417f4b806ec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.593624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deed7c-532f-449c-b840-8417f4b806ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.594007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30deed7c-532f-449c-b840-8417f4b806ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.610052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deed7c-532f-449c-b840-8417f4b806ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:56.613138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30deed7c-532f-449c-b840-8417f4b806ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-30deed7c-532f-449c-b840-8417f4b806ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:56.638040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:56.638998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:56.640022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:56.640022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:56.640022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:56.640022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:56.642907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 354/712] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:56.645423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a655-7474-46e3-9aca-896270328e71 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.645926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a655-7474-46e3-9aca-896270328e71 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.646864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34a655-7474-46e3-9aca-896270328e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.651552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f584a3-0c5e-4e27-b63c-4fbd5b30e3f6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:56.668734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fe34a655-7474-46e3-9aca-896270328e71 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:56.670764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe34a655-7474-46e3-9aca-896270328e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:56.699549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:56.700726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.701023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.701307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.702705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=ee056b132bd24851bd0e2afe9cb7170e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:56.703037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:56.704489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:56.713442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 359/713] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 15:59:56 2022] GET /identity/v3/role_assignments?user.id=ee056b132bd24851bd0e2afe9cb7170e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:56.715123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:56.773130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 355/714] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:56.795619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34a655-7474-46e3-9aca-896270328e71 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:56.827455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:56.861001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a8535-00bc-4b8e-8d46-d5c66e3541fc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:56.865575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:56.916484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:56.917624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:56.918231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:56.918661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:56.919568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:56.920116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:56.922284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:56.939614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 360/715] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:56.945530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:57.005813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34a655-7474-46e3-9aca-896270328e71 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.051042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.052389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.052906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.053336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.054475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.056919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:57.077873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:57.079672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 15:59:57.081057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:57.084766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 356/716] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 15:59:56 2022] GET /identity/v3/services/network => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:57.090978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:57.196834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3deee-c194-498c-aee1-04d5cee06bce None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.261433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.262238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.262556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.262838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.263564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.265258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.299872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 361/717] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 15:59:56 2022] POST /identity/v3/users => generated 254 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:57.302344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:57.318228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f18e-4143-4e05-879b-a495bca5f233 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.364219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f3a0f-1f65-469f-bfda-b914a549d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.364935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f3a0f-1f65-469f-bfda-b914a549d602 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.365218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f3a0f-1f65-469f-bfda-b914a549d602 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.365494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f3a0f-1f65-469f-bfda-b914a549d602 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.366288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f3a0f-1f65-469f-bfda-b914a549d602 None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:57.366644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f3a0f-1f65-469f-bfda-b914a549d602 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.368244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f3a0f-1f65-469f-bfda-b914a549d602 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.376942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 357/718] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:57.380625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:57.408754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:57.507950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d635e3b-3d2e-40d5-9d70-dd0cfb609fb4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:57.547918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cb8f1-5a31-40eb-8db9-868ef5974537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:57.548613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cb8f1-5a31-40eb-8db9-868ef5974537 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:57.548949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cb8f1-5a31-40eb-8db9-868ef5974537 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:57.549281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cb8f1-5a31-40eb-8db9-868ef5974537 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:57.550089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cb8f1-5a31-40eb-8db9-868ef5974537 None admin] RBAC: Adding query filter params (type=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:57.550425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cb8f1-5a31-40eb-8db9-868ef5974537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:57.552153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cb8f1-5a31-40eb-8db9-868ef5974537 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:57.558625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 362/719] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 15:59:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:58.083393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0e01f-4fdd-434e-af38-a680ec3f7870 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:58.083778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0e01f-4fdd-434e-af38-a680ec3f7870 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:58.083963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0e01f-4fdd-434e-af38-a680ec3f7870 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:58.085588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 358/720] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:58.091370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708521ab-d5ca-4887-820d-a58940634851 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:58.091820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708521ab-d5ca-4887-820d-a58940634851 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:58.092009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708521ab-d5ca-4887-820d-a58940634851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:58.109167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-708521ab-d5ca-4887-820d-a58940634851 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:58.110563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-708521ab-d5ca-4887-820d-a58940634851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:58.196299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 363/721] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:58.232676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0e01f-4fdd-434e-af38-a680ec3f7870 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:58.379915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0e01f-4fdd-434e-af38-a680ec3f7870 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:58.386618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251db60-6dd3-46ad-9320-2fdd5be6f2b0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:58.390142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251db60-6dd3-46ad-9320-2fdd5be6f2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:58.390439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251db60-6dd3-46ad-9320-2fdd5be6f2b0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:58.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 364/722] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 15:59:58.402657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece3eaa-4810-499a-b9b7-89282fb28451 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:58.402953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece3eaa-4810-499a-b9b7-89282fb28451 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:58.403236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ece3eaa-4810-499a-b9b7-89282fb28451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:58.422994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:58.424030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:58.424524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:58.425011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:58.426181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:58.427783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:58.438799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2ece3eaa-4810-499a-b9b7-89282fb28451 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:58.440014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2ece3eaa-4810-499a-b9b7-89282fb28451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:58.445962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 359/723] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 15:59:58 2022] POST /identity/v3/services => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 15:59:58.455131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:58.551210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 365/724] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:58.587113 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:58.608335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:58.732631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0c60-0aeb-4789-9f8b-bddeb6b4da3e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:58.776796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:58.777793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:58.778152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:58.778475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:58.791664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:58.800777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:58.802783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:58.804152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 360/725] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:58.811116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece3eaa-4810-499a-b9b7-89282fb28451 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:58.939503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ece3eaa-4810-499a-b9b7-89282fb28451 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:58.987034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:58.987748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:58.988027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:58.988300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:58.989570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:58.989904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:58.991392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:59.007253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 366/726] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 15:59:59.012786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:59.103362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:59.135917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae334f-1713-4b18-be49-3fde769b6e66 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:59.173155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:59.174000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:59.174289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:59.174568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:59.186721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:59.188444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 15:59:59.206951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:59.208885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:59.210141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 15:59:59.210141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 15:59:59.210141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 15:59:59.210141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:59.210141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 361/727] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 15:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:59.213837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:59.220406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:59.359563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b201-36c5-4a9e-a565-f888edf18e57 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:59.401653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c47cab-40b8-4b1b-a419-4d81e89c3f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:59.402468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c47cab-40b8-4b1b-a419-4d81e89c3f2d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:59.402750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c47cab-40b8-4b1b-a419-4d81e89c3f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:59.403070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c47cab-40b8-4b1b-a419-4d81e89c3f2d None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:59.403865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c47cab-40b8-4b1b-a419-4d81e89c3f2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:59.404243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c47cab-40b8-4b1b-a419-4d81e89c3f2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:59.405915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c47cab-40b8-4b1b-a419-4d81e89c3f2d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:59.424933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 367/728] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 15:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 15:59:59.429969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:59.557358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d47b-e2c0-48e7-bb9a-b2cd26f349fd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:59.557708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d47b-e2c0-48e7-bb9a-b2cd26f349fd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:59.557949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361d47b-e2c0-48e7-bb9a-b2cd26f349fd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:59.559769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 368/729] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 15:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 15:59:59.565056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259832-16ec-4438-9565-e3e701d24d21 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:59.565579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259832-16ec-4438-9565-e3e701d24d21 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:59.566116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13259832-16ec-4438-9565-e3e701d24d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:59.585744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a769c8-1e5f-4aba-82eb-7e721b0c898e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:59.602296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-13259832-16ec-4438-9565-e3e701d24d21 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 15:59:59.604070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-13259832-16ec-4438-9565-e3e701d24d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 15:59:59.614543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:59.630453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:59.631284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:59.631575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:59.631855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:59.641134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:59.642657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 15:59:59.668396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:59.669407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 15:59:59.670382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 15:59:59.675378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 362/730] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 15:59:59.682452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:59.694845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 15:59:59.708343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 369/731] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 15:59:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 15:59:59.731639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259832-16ec-4438-9565-e3e701d24d21 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 15:59:59.898033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9a77-823c-4cfd-ba07-bb51bd36994c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:59.911857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13259832-16ec-4438-9565-e3e701d24d21 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 15:59:59.962942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:59.963759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:59.964038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:59.964311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:59.977462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 15:59:59.982466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 15:59:59.983290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 15:59:59.983579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 15:59:59.983935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 15:59:59.984895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 15:59:59.985326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 15:59:59.985433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 15:59:59.986358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 15:59:59.987188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 370/732] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 15:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 15:59:59.987904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 15:59:59.998753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.036596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 363/733] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 15:59:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:00.043518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.074250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.126942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.143872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7780f-5ec3-40eb-aac9-399876772906 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.189561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58622a27-831e-4bc7-b83a-4408c04ef51d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.219563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.220542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.220998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.221207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.222153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:00.222587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.224902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.241497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 371/734] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 15:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:00.242570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025f010-a5b4-4e18-b4e2-8f27ea12f85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.243477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025f010-a5b4-4e18-b4e2-8f27ea12f85e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.243995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025f010-a5b4-4e18-b4e2-8f27ea12f85e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.244509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025f010-a5b4-4e18-b4e2-8f27ea12f85e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.248648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.324841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025f010-a5b4-4e18-b4e2-8f27ea12f85e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.326701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025f010-a5b4-4e18-b4e2-8f27ea12f85e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.382254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.402979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f378c-da94-4f5d-bc24-4f14cd3912bb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.424225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.454472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230c60c-a894-491b-bdcd-8bac82c7836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.455204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230c60c-a894-491b-bdcd-8bac82c7836a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.455489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230c60c-a894-491b-bdcd-8bac82c7836a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.455766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230c60c-a894-491b-bdcd-8bac82c7836a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.457862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 364/735] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 16:00:00.459737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c37cb-79ea-4332-8881-e9bcedfdb523 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.460268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c37cb-79ea-4332-8881-e9bcedfdb523 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.460746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c37cb-79ea-4332-8881-e9bcedfdb523 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.462494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 365/736] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:00.463403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c37cb-79ea-4332-8881-e9bcedfdb523 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.474412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230c60c-a894-491b-bdcd-8bac82c7836a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.476611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230c60c-a894-491b-bdcd-8bac82c7836a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0230c60c-a894-491b-bdcd-8bac82c7836a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:00.503496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:00.505430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:00.506775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:00.506775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:00.506775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:00.506775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:00.506775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 372/737] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:00.507273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee7417-f9c1-45f9-afc2-128d1c011902 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.507524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee7417-f9c1-45f9-afc2-128d1c011902 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.507808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee7417-f9c1-45f9-afc2-128d1c011902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.526795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-38ee7417-f9c1-45f9-afc2-128d1c011902 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:00.528375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-38ee7417-f9c1-45f9-afc2-128d1c011902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:00.597585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c37cb-79ea-4332-8881-e9bcedfdb523 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.605748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 373/738] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 16:00:00.605748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee7417-f9c1-45f9-afc2-128d1c011902 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.650325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5014195-df15-4daf-a2e6-4cea9e9960ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.651408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014195-df15-4daf-a2e6-4cea9e9960ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.651408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014195-df15-4daf-a2e6-4cea9e9960ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.651685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5014195-df15-4daf-a2e6-4cea9e9960ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.663747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5014195-df15-4daf-a2e6-4cea9e9960ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.665202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5014195-df15-4daf-a2e6-4cea9e9960ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.667027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f5014195-df15-4daf-a2e6-4cea9e9960ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:00.683062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:00.684000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:00.684920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:00.684920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:00.684920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:00.684920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:00.684920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 366/739] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:00.688926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539552-d6e0-4309-8d11-820a485a7237 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.688926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539552-d6e0-4309-8d11-820a485a7237 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.688926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e539552-d6e0-4309-8d11-820a485a7237 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.693822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 367/740] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:00.693822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539552-d6e0-4309-8d11-820a485a7237 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.714853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.722772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee7417-f9c1-45f9-afc2-128d1c011902 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.767785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.768465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.768752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.769031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.769841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:00.770156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.771549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.792032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 374/741] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:00.792958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.806262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e539552-d6e0-4309-8d11-820a485a7237 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.843347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2aadd-ecad-4590-9e23-bfe0c27255ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.843926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2aadd-ecad-4590-9e23-bfe0c27255ce None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.844208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2aadd-ecad-4590-9e23-bfe0c27255ce None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.844482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2aadd-ecad-4590-9e23-bfe0c27255ce None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.845204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2aadd-ecad-4590-9e23-bfe0c27255ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.846610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2aadd-ecad-4590-9e23-bfe0c27255ce None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.861945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 368/742] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:00 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:00.863189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127d5f7-a20d-403b-bd06-38475b2a2fd6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.863559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127d5f7-a20d-403b-bd06-38475b2a2fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.864329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127d5f7-a20d-403b-bd06-38475b2a2fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.882506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2127d5f7-a20d-403b-bd06-38475b2a2fd6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:00.883798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2127d5f7-a20d-403b-bd06-38475b2a2fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:00.924029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddc809-c202-4630-83b5-9899824c0ea1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:00.941782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:00.980347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:00.980347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:00.980347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:00.980347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:00.980722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:00.980722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:00.980722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:00.981402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 369/743] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:00.982689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127d5f7-a20d-403b-bd06-38475b2a2fd6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:00.998693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 375/744] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:01.000927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.024103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:01.131858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127d5f7-a20d-403b-bd06-38475b2a2fd6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.152249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccc771-e45d-47fa-aad0-0de833fe7ddc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.182445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.183254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.183543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.183827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.192347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.193858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.201286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.202627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.203247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.203794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:01.217595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:01.218613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:01.219899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 370/745] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:01.220517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.234021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.236276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:01.255549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:01.256497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:01.257392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:01.260546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 376/746] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:01.265826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.271395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:01.333925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3371b4b-fb52-4da8-bd25-48d58fb9e2f2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.347282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:01.371758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.372657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.372838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.373125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.384096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d455752-bbc9-4542-9908-809f20b47a2f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.384731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:01.393243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:01.394201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:00:01.395107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:01.399369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 371/747] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:01.400256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.425648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.426875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.427462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.428087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.429254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:01.429794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.431736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.455477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 377/748] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:01.456853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.518602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:01.522501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2865d2-b24d-465d-9c97-18ed664026cd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.559428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.560110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.560389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.560672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.561466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:01.561813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.563553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.576607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 372/749] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:01.577207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.585836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412152f-72f3-45c5-8b0a-33a4ec0b0275 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.617174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:01.624337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9955497-3bd1-4e28-9763-cf5590a2eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.625080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9955497-3bd1-4e28-9763-cf5590a2eb9c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.625367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9955497-3bd1-4e28-9763-cf5590a2eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.625644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9955497-3bd1-4e28-9763-cf5590a2eb9c None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.626419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9955497-3bd1-4e28-9763-cf5590a2eb9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:01.626728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9955497-3bd1-4e28-9763-cf5590a2eb9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.628104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9955497-3bd1-4e28-9763-cf5590a2eb9c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.637524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 378/750] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:01.638814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebfb4d-6b56-4c7a-8624-465e899a4b23 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.639104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebfb4d-6b56-4c7a-8624-465e899a4b23 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.639384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebfb4d-6b56-4c7a-8624-465e899a4b23 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.645802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 379/751] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:01.645802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebfb4d-6b56-4c7a-8624-465e899a4b23 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.694497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1a3e3-5938-4100-96cd-e9f99f357f12 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.733548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.734062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.734305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.747085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.748727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.763031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebfb4d-6b56-4c7a-8624-465e899a4b23 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:01.769217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:01.770341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:01.771439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:01.771439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:01.771439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:01.771439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:01.775610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 373/752] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:01.776458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.792862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:01.806273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f4773-8216-40ff-850a-aa8b777475af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.807005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f4773-8216-40ff-850a-aa8b777475af None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.807433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f4773-8216-40ff-850a-aa8b777475af None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.807687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f4773-8216-40ff-850a-aa8b777475af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.808584 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f4773-8216-40ff-850a-aa8b777475af None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=9b4b1f0775ba4875a550ea986ed483e7) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:01.808924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f4773-8216-40ff-850a-aa8b777475af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.810596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f4773-8216-40ff-850a-aa8b777475af None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:01.816724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 380/753] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/role_assignments?user.id=9b4b1f0775ba4875a550ea986ed483e7&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:01.818822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de67c4f-f827-4e46-b620-9ceb7bb00f13 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.818822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de67c4f-f827-4e46-b620-9ceb7bb00f13 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.818822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de67c4f-f827-4e46-b620-9ceb7bb00f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.855975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1de67c4f-f827-4e46-b620-9ceb7bb00f13 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:01.857295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1de67c4f-f827-4e46-b620-9ceb7bb00f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:01.904828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda5982-37ee-4757-b461-8e896e42bfb8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:01.928690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:01.947582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 381/754] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:01.948739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de67c4f-f827-4e46-b620-9ceb7bb00f13 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:01.960222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:01.961226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:01.961665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:01.962007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:01.985277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:01.987115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:02.015321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:02.016290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:02.017179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:02.017179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:02.017179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:02.017179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:02.017179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 374/755] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:02.018643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.074117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de67c4f-f827-4e46-b620-9ceb7bb00f13 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.091584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:02.118405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.119430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.119903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.120365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.121300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:02.121911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.123473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.135510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57580f-f5a5-4e42-b83c-a2ce6c798715 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.141106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 382/756] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:02.142215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.179323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.180402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.180794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.181175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.181971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.183431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:02.197223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:02.198268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:00:02.199276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:02.205275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 375/757] 10.222.0.88 () {56 vars in 1129 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:02.206542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.220076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:02.283080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755eeec-4990-4b69-9271-67597a9cca1b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.318203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e13e97-5509-4b9c-a146-ea8896d86fc2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.322339 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.323256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.323754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.324278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.325209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:02.325704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.327388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.342086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 383/758] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:02.343189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.350387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:02.360111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.361057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.361445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.361850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.370281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.371872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:02.391820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:02.392764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:02.393678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:02.397789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 376/759] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:02.398829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.460838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e2cd5-1c98-485d-a6ed-8db38a5226ae None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.469529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:02.506913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.507655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.507942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.508222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.509227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:02.509405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.510797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.516293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 384/760] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:02.517262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.532780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df841741-a19f-4c14-87a5-f7b0c9f46909 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.575233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.575972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.576258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.576539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.587023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.588732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:02.608984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:02.610090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:02.611037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:02.614055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 377/761] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:02.615110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.641931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bf977-9ad3-48cb-9e57-df761482f4df None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.679921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:02.689150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.690637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.691288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.691864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.693310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:02.694128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.696249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.716071 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 385/762] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:02.717565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.744227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bae30f-4143-46c9-aa95-c3f2a9921542 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.756064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:02.788921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.790149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.790680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.791222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.792298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:02.792839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.794513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.801930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 378/763] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:02.802970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.832677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3c716-ab6c-44bf-abcc-d9f93a17d897 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.869328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.870029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.870313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.870589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:02.871423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:02.871735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:02.873158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:02.898013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 386/764] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:02.898921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:02.946253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b41960-06c6-40ee-a584-aa0deb6c0def None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:02.968507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:02.996999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:02.998022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:02.998377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:02.998724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/ee056b132bd24851bd0e2afe9cb7170e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.001045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:03.002763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=ee056b132bd24851bd0e2afe9cb7170e, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.005057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.016523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623dcbb-b93d-4880-baae-ed2becd661d7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:03.027096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 379/765] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:02 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/ee056b132bd24851bd0e2afe9cb7170e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:03.028074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:03.072047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:03.072804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.073096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.073383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.075932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] RBAC: Adding query filter params (interface=public, service_id=2ee3f743119c46a1a10eb30e4ba98aad, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:03.075932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.076407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.088242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 387/766] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:00:02 2022] GET /identity/v3/endpoints?service_id=2ee3f743119c46a1a10eb30e4ba98aad&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:03.094227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:03.237620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d8c93-0401-42f9-8dd2-0a2a7c9ac151 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:03.280550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdaaf47-feb9-4613-94e8-67ca4baff0ac None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:03.282385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:03.283228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.283516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.283795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.322287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1654ca0-b307-430c-8cb6-a5db0d5b4baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:03.323745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1654ca0-b307-430c-8cb6-a5db0d5b4baa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.324264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1654ca0-b307-430c-8cb6-a5db0d5b4baa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.324765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1654ca0-b307-430c-8cb6-a5db0d5b4baa None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.325736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1654ca0-b307-430c-8cb6-a5db0d5b4baa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.327827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1654ca0-b307-430c-8cb6-a5db0d5b4baa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.339701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:03.340140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 388/767] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:03 2022] GET /identity/v3/services => generated 772 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:03.342290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33036f8d-ecd8-4f7f-a328-5f43c7e35659 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.342786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33036f8d-ecd8-4f7f-a328-5f43c7e35659 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.343263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33036f8d-ecd8-4f7f-a328-5f43c7e35659 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.348593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 389/768] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:03.351966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e51b4-7b2b-4cc7-b15c-1c6cd3503425 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.352511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e51b4-7b2b-4cc7-b15c-1c6cd3503425 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.352988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e51b4-7b2b-4cc7-b15c-1c6cd3503425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.376625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.378302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.386851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-828e51b4-7b2b-4cc7-b15c-1c6cd3503425 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:03.388519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-828e51b4-7b2b-4cc7-b15c-1c6cd3503425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:03.412037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:03.557497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 390/769] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:03.571755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 380/770] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:03.577023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e51b4-7b2b-4cc7-b15c-1c6cd3503425 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:03.589156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:03.684598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:03.721661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e51b4-7b2b-4cc7-b15c-1c6cd3503425 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:03.741643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060a802-bc47-40bc-9764-d82d0d369773 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:03.792869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:03.793988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.794442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.794766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.815155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:03.815973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:03.816260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:03.816541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] PATH_INFO: `/v3/services/network` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:03.817728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.819296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.826626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:03.828112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:03.834348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:03.835335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:03.836150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 381/771] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:03 2022] GET /identity/v3/services/network => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:03.845075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:03.850116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:03.852503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:03.853396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:03.854337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:03.854337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:03.854337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:03.854337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:03.854990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 391/772] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:03.864591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.025235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20a988-66e5-4171-ad35-ca53b1181aba None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.053073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfaaa70-8597-4713-ab3c-10fc67cf3394 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.096387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.097111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.097448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.097747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.098872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:04.099572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.101918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.117270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 392/773] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:04.122909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.135362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.136495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.136997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.137481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.138515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:04.139049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.141067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.146941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:04.157999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 382/774] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:04.163194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.239334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:04.292956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78093ee6-5d36-459f-88b0-47d87295fbd0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.294369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17559dc7-ce4f-4276-b8bf-244364f487f8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.329946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f30f27-3864-4be3-84c3-ebab75fac2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.330811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f30f27-3864-4be3-84c3-ebab75fac2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.331099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f30f27-3864-4be3-84c3-ebab75fac2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.331428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f30f27-3864-4be3-84c3-ebab75fac2c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.338630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee68d5-a7a7-4603-a408-7d608efed433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.338990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f30f27-3864-4be3-84c3-ebab75fac2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.339862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee68d5-a7a7-4603-a408-7d608efed433 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.340350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee68d5-a7a7-4603-a408-7d608efed433 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.340470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f30f27-3864-4be3-84c3-ebab75fac2c9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.341402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee68d5-a7a7-4603-a408-7d608efed433 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.342474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee68d5-a7a7-4603-a408-7d608efed433 None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:04.343004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee68d5-a7a7-4603-a408-7d608efed433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.344666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee68d5-a7a7-4603-a408-7d608efed433 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.352041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 383/775] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:04.354291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e747-261a-4af1-8dfa-3dbe6802b915 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.354789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e747-261a-4af1-8dfa-3dbe6802b915 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.355271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e747-261a-4af1-8dfa-3dbe6802b915 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.357027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 384/776] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:04.359143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505e747-261a-4af1-8dfa-3dbe6802b915 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-36f30f27-3864-4be3-84c3-ebab75fac2c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:04.360912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:04.361817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:04.362753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:04.367260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 393/777] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:04.368634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955997fd-ef72-40ba-acbc-1c7c71afb941 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.368927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955997fd-ef72-40ba-acbc-1c7c71afb941 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.369212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955997fd-ef72-40ba-acbc-1c7c71afb941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.386814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-955997fd-ef72-40ba-acbc-1c7c71afb941 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:04.388069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-955997fd-ef72-40ba-acbc-1c7c71afb941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:04.434923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:04.459157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 394/778] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:04.460124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955997fd-ef72-40ba-acbc-1c7c71afb941 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.484188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505e747-261a-4af1-8dfa-3dbe6802b915 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.511281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:04.526666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.527719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.528212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.528696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.529825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] RBAC: Adding query filter params (interface=public, service_id=a8d8906375d64ea29ed13fc18d628d5f, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:04.530347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.532036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.541241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 385/779] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/endpoints?service_id=a8d8906375d64ea29ed13fc18d628d5f&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:04.542345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.583923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955997fd-ef72-40ba-acbc-1c7c71afb941 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.628378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.629938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.630450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.631333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.632653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:04.633169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.635633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.650709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 395/780] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:04.652433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.699686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4059f-3439-47d6-8eb4-a6fe1b21005c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.730410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:04.739759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.740857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.741355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.741900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.753207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:04.761824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:04.762760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:04.763591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:04.766622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 386/781] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:04.767725 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.774441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e06a7-e43c-404a-a827-1e74e565c2d1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.785300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:04.812685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.813388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.813959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.814095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.814746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.816239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.826074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 396/782] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:04.827113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:04.887290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf123e22-335b-4ae2-b372-79263ad6858b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:04.929260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:04.931219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:04.932111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:04.932951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:04.935131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=bf00375b0f334f7bada8f21ea3766ac6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:04.936066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:04.938019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:04.945524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 387/783] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/role_assignments?user.id=bf00375b0f334f7bada8f21ea3766ac6&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:05.030094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1db3d-b9e9-47bf-965b-20bdce19c6fa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:05.091617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:05.092665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.093178 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.093778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.094809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:05.095330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:05.096937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:05.107997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 397/784] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:05.114542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:05.190248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:05.244249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aff334-bfda-4005-87df-46b0c7cf3b8e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:05.282057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768c87a-0fe3-461a-b0bc-ad5e6260c108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:05.282866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c87a-0fe3-461a-b0bc-ad5e6260c108 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.283152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c87a-0fe3-461a-b0bc-ad5e6260c108 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.283429 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768c87a-0fe3-461a-b0bc-ad5e6260c108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.296135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768c87a-0fe3-461a-b0bc-ad5e6260c108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:05.297757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768c87a-0fe3-461a-b0bc-ad5e6260c108 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0768c87a-0fe3-461a-b0bc-ad5e6260c108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:05.315832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:05.317316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:05.318228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:05.318228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:05.318228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:05.318228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:05.318228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 388/785] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:05.323420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:05.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:05.446528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223b79-2ce6-4535-864f-361c03575d77 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.446791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223b79-2ce6-4535-864f-361c03575d77 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.447018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223b79-2ce6-4535-864f-361c03575d77 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.448745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 389/786] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:05.458545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68726-a500-4dc1-9ffa-8bfef4cc0560 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.458839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68726-a500-4dc1-9ffa-8bfef4cc0560 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.459125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a68726-a500-4dc1-9ffa-8bfef4cc0560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.462361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934b1a0-4230-403b-998c-2595d8913aec None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:05.487475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-52a68726-a500-4dc1-9ffa-8bfef4cc0560 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:05.489219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-52a68726-a500-4dc1-9ffa-8bfef4cc0560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:05.530478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:05.531162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.531514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.531801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.532535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:05.532849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:05.534270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:05.553423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 398/787] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:05.554946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:05.620687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 390/788] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:05.655276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68726-a500-4dc1-9ffa-8bfef4cc0560 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:05.688999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:05.693311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:05.701655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e9a08-ca15-46d6-8bfe-fb9413ac8b62 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:05.757312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:05.758153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.758434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.758706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.772691 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a68726-a500-4dc1-9ffa-8bfef4cc0560 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:05.778558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:05.780023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:05.805920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:05.806959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:00:05.807934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:05.822701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 399/789] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:05.823527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:05.824567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:05.824952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:05.825336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:05.827319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:05.851189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:05.858958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:05.859975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:05.860825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:05.865061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 391/790] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:05.866303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:05.978842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b1ee4-3eac-4b84-b09b-5ce6d088cf2a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:05.982044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.043712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.044408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.044697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.044980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.045766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:06.046084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.047560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.067422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 392/791] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:06.073375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.091338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f2d56-a659-43dc-9815-ad1438794037 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.158644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5519213-2654-43b9-9396-afc43e8be3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.159327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519213-2654-43b9-9396-afc43e8be3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.159614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519213-2654-43b9-9396-afc43e8be3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.159898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5519213-2654-43b9-9396-afc43e8be3a5 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.161392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5519213-2654-43b9-9396-afc43e8be3a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:06.161708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5519213-2654-43b9-9396-afc43e8be3a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.163231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5519213-2654-43b9-9396-afc43e8be3a5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.181414 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 400/792] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 16:00:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:06.182851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84750df-b3fe-4752-82d6-8b0cdac73ad7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.183148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84750df-b3fe-4752-82d6-8b0cdac73ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.183430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84750df-b3fe-4752-82d6-8b0cdac73ad7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.185032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 401/793] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:06.192780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84750df-b3fe-4752-82d6-8b0cdac73ad7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.209217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.221618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f431e4f-60a3-4dca-8923-3251292f85c9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.260080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956e00a-9981-46e1-aa4e-45dbc8d6f2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.261345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956e00a-9981-46e1-aa4e-45dbc8d6f2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.261824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956e00a-9981-46e1-aa4e-45dbc8d6f2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.262301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956e00a-9981-46e1-aa4e-45dbc8d6f2e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.274874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956e00a-9981-46e1-aa4e-45dbc8d6f2e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.276543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956e00a-9981-46e1-aa4e-45dbc8d6f2e3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5956e00a-9981-46e1-aa4e-45dbc8d6f2e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:06.295668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:06.296878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:06.298070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:06.298070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:06.298070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:06.298070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:06.304992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 393/794] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:06.306571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf23a-25f4-4b9d-8379-797e2adbe45f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.306691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84750df-b3fe-4752-82d6-8b0cdac73ad7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.307130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf23a-25f4-4b9d-8379-797e2adbe45f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.307559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcf23a-25f4-4b9d-8379-797e2adbe45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.326861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8ebcf23a-25f4-4b9d-8379-797e2adbe45f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:06.328479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ebcf23a-25f4-4b9d-8379-797e2adbe45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:06.333282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.351763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.352894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.353250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.353490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.365913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.367497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:06.388160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:06.389325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:06.390559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:06.390559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:06.390559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:06.390559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:06.390559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 402/795] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:06.390559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.416469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 394/796] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:06.417623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf23a-25f4-4b9d-8379-797e2adbe45f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.482709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.523494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e24a-8148-4d67-8097-5b4c3dff75ba None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.534129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcf23a-25f4-4b9d-8379-797e2adbe45f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.560539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.561386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.561820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.562206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.563035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:06.563450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.565029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.574080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.575030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.575521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.575979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.576880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:06.577368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.582426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 403/797] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:06.582426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.583776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.594194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.601205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 395/798] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:06.602372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.684692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e130dda-8ea4-4c45-98f5-21d465322322 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.709117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.713142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a94b0b-feda-4129-89e4-197265725079 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.720585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.721484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.721873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.722591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.723547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.725017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:06.738836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:06.739813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 16:00:06.740662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:06.745211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 404/799] 10.222.0.88 () {56 vars in 1129 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:06.746131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.757181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.758385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.758853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.759307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.767890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.769695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:06.785085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:06.787090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:06.788066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:06.795232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 396/800] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:06.796400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.859634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e9674-fc38-474f-8d7e-b1b4b86c81fb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.861844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:06.898838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.899794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.900155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.900509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.908392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.909986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.912038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993d493-76fb-4a73-8a96-4756c7b07947 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:06.926183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:06.927159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:06.928268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 405/801] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:06.928981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.952521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:06.953982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:06.954427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:06.954892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:06.955891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:06.956383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:06.958596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:06.964436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 397/802] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:06.965547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:06.989574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.046703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1251-a0ec-4d60-a165-5fac9a78af72 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.084161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aca1e2-aceb-440c-94c0-a76d6629df22 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.087525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.088207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.088500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.088783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.090095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:07.090665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.092870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.110811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.118206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 406/803] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:07.119243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:07.126904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.127965 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.128459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.128939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.130033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:07.130601 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.132368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.147730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 398/804] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:07.148989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:07.227633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b726c64-c912-4617-96bd-1051a7613868 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.235460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.265145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.265798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.266088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.266371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.267088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:07.267398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.268785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.269664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79acb43-0916-4d37-8639-23f376220299 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.274583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 407/805] 10.222.0.88 () {56 vars in 1124 bytes} [Thu Jun 16 16:00:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:07.275360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:07.309083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b59aaf-27bc-4f9e-8700-a01c549cf42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.310122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b59aaf-27bc-4f9e-8700-a01c549cf42f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.310712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b59aaf-27bc-4f9e-8700-a01c549cf42f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.311236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b59aaf-27bc-4f9e-8700-a01c549cf42f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.368588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.386598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b8da5-691e-490d-be44-855f89dcc0db None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.396695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b59aaf-27bc-4f9e-8700-a01c549cf42f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.398845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b59aaf-27bc-4f9e-8700-a01c549cf42f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.425056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.426039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.426337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.426624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/9b4b1f0775ba4875a550ea986ed483e7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.428608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=9b4b1f0775ba4875a550ea986ed483e7, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.430087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.446666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 408/806] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:07 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/9b4b1f0775ba4875a550ea986ed483e7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:07.447868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:07.500711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.540004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 399/807] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:07.541893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330724b-cf32-40aa-8789-21f623ebe9b9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.542459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330724b-cf32-40aa-8789-21f623ebe9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.542943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330724b-cf32-40aa-8789-21f623ebe9b9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.545027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 400/808] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:07.546290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330724b-cf32-40aa-8789-21f623ebe9b9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:07.599682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2e0c3-1231-46d8-aa1a-10089bd60e85 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.643489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b453c-c103-468b-837f-8cc49e7950eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.644304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b453c-c103-468b-837f-8cc49e7950eb None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.644563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b453c-c103-468b-837f-8cc49e7950eb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.644865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b453c-c103-468b-837f-8cc49e7950eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.645617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b453c-c103-468b-837f-8cc49e7950eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.647045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b453c-c103-468b-837f-8cc49e7950eb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.653094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.673912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 409/809] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:07.675962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13166e11-7b1b-40b6-adc3-b5b514c46ce3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.676862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13166e11-7b1b-40b6-adc3-b5b514c46ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.677155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13166e11-7b1b-40b6-adc3-b5b514c46ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.683991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330724b-cf32-40aa-8789-21f623ebe9b9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.700445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-13166e11-7b1b-40b6-adc3-b5b514c46ce3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:07.702227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-13166e11-7b1b-40b6-adc3-b5b514c46ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:07.725904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf0e43-47d6-425c-8d5e-265d865eeb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.726716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf0e43-47d6-425c-8d5e-265d865eeb1c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.727001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf0e43-47d6-425c-8d5e-265d865eeb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.727279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf0e43-47d6-425c-8d5e-265d865eeb1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.739711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf0e43-47d6-425c-8d5e-265d865eeb1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.741512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf0e43-47d6-425c-8d5e-265d865eeb1c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d5bf0e43-47d6-425c-8d5e-265d865eeb1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:07.765418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:07.766678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:07.768770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:07.768770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:07.768770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:07.768770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:07.768770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 401/810] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:07.769075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43e93-397f-4508-9928-d063bf5e1ae3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.769337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43e93-397f-4508-9928-d063bf5e1ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.769543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece43e93-397f-4508-9928-d063bf5e1ae3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.775572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 402/811] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:07.775572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece43e93-397f-4508-9928-d063bf5e1ae3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:07.807113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 410/812] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:07.808515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc6ad5-4262-49a9-a06d-b76cbf561be2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.808807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc6ad5-4262-49a9-a06d-b76cbf561be2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.809085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc6ad5-4262-49a9-a06d-b76cbf561be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.829833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d4cc6ad5-4262-49a9-a06d-b76cbf561be2 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:07.831625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d4cc6ad5-4262-49a9-a06d-b76cbf561be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:07.847734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.913436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece43e93-397f-4508-9928-d063bf5e1ae3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:07.932944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 411/813] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:07.933615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc6ad5-4262-49a9-a06d-b76cbf561be2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:07.958107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:07.975265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:07.975950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:07.976237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:07.976514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:07.977235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:07.977543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:07.979073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:07.996864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 403/814] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:07.998473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.064641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc6ad5-4262-49a9-a06d-b76cbf561be2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.111558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.112555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.112914 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.113295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] PATH_INFO: `/v3/services/network` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.114108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.115736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:08.128680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:08.130417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:08.131390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 412/815] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:07 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:08.132365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.138047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773dec0-988d-4b45-b829-447a5d232fc1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.157943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.178888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.179707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.179973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.180244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.196977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:08.209345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:08.210370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:00:08.211222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:08.219900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 404/816] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 16:00:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:08.219900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.278240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.340489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb650b-c4d7-44b4-b4f6-9dcc4a507690 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.379981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.380554 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.380841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.381121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.381875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:08.382191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.383599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.393374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 405/817] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:08.394402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.407974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f5dc6-a759-4eab-978f-418a7ca51431 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.448333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.449508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.450138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.450762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.459546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.461514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:08.477829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:08.478761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:08.479688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:08.482134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 413/818] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:08.483218 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.501963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0326ed-6b4c-48a9-87ce-5b332278032c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.516729 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.538970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.539673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.539936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.540221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.541036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:08.541281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.542702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.556719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 406/819] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:08.557582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.573591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.614783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e00ea1-7a89-4caa-90d5-344caff6c3b9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.658723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.659796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.660283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.660915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.662005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] RBAC: Adding query filter params (type=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:08.662550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.664302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.667548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79117f1-d2c4-48a0-ab2e-8647d4d8dae3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.670889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 414/820] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:08.671990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.709541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.710167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.710460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.710744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.711532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:08.711850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.713303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.727530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 407/821] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:08.728463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.764793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.798498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0936f0f-8da6-4453-a375-3c92e8e24b13 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.834142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e20c2-87b1-425e-b077-cf6dbbd38258 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:08.840159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.841552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.842075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.842659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.845038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:08.859550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.861518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.872159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:08.872857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:08.873109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:08.873595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:08.874424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:08.876098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:08.885237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:08.886376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:08.887416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:08.887416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:08.887416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:08.887416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:08.891681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 415/822] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:08.893271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:08.910041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 408/823] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:08.911592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.032123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.051896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e697c2-eba0-42dd-9c6f-57c937ad8095 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.060310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ffbfe-ba05-4e68-8de0-738ce3b759ef None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.092734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.093435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.093745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.094069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.094926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:09.095248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.096675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.113092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 409/824] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:09.119456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.162622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.163400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.163689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.163970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.164809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=ee056b132bd24851bd0e2afe9cb7170e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:09.165125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.166705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.179906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 416/825] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:08 2022] GET /identity/v3/role_assignments?user.id=ee056b132bd24851bd0e2afe9cb7170e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:09.247105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426dee7-7f32-4c93-b488-a342817c9400 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.289326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.290112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.290396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.290675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.299264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.300753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:09.326010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:09.327035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:09.327978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:09.330783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 410/826] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:09.342880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.409989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.472821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec4ec4-96c1-4c1e-95f3-d90b0c5cb033 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.512436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.513130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.513428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.513661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.514562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:09.514883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.516325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.537375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 417/827] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:09.542618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.556021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.653712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a84274-85b4-4155-9950-fa2a0ae4caab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.692282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c0847-1edd-4a58-bc14-4c077820839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.693083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0847-1edd-4a58-bc14-4c077820839b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.693366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0847-1edd-4a58-bc14-4c077820839b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.693639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c0847-1edd-4a58-bc14-4c077820839b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.704509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c0847-1edd-4a58-bc14-4c077820839b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.706016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c0847-1edd-4a58-bc14-4c077820839b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2f9c0847-1edd-4a58-bc14-4c077820839b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:09.727416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:09.728395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:09.729290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:09.729290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:09.729290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:09.729290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:09.729290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 411/828] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:09.734904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.791229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1536964-c80e-43d9-bd86-c1b7bdc4f32d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.791557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1536964-c80e-43d9-bd86-c1b7bdc4f32d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.791787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1536964-c80e-43d9-bd86-c1b7bdc4f32d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.793343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 412/829] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:09.798917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06c676-0838-452b-88c2-9030b9771699 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.799502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06c676-0838-452b-88c2-9030b9771699 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.799995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06c676-0838-452b-88c2-9030b9771699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.817775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fe06c676-0838-452b-88c2-9030b9771699 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:09.819296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe06c676-0838-452b-88c2-9030b9771699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:09.849991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ab6b7-391c-482e-bfed-a35e66617d2a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:09.860098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.888059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:09.888731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:09.889015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:09.889293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:09.890062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:09.890346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:09.891805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:09.893735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:09.909142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 418/830] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:09.911532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:09.913189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 413/831] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:09.941159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06c676-0838-452b-88c2-9030b9771699 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.047137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cd638-5ca9-4781-a9da-8a1d8ef1454a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.052830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06c676-0838-452b-88c2-9030b9771699 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.089326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.090174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.090456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.090729 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.092351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.093501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.094108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.094411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.107177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.107911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:10.109320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:10.116686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:10.117664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:10.118565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:10.122019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 414/832] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:10.124060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:10.129851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:10.130828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:10.131768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:10.140413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 419/833] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:10.148946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.190309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:10.194218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:10.251369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae56eb-787d-4537-842a-432e4d0ed424 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.278688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c88e3-032f-47e2-9f21-8fd19ca553d6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.297667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fa071d-3abf-4ff0-a8aa-8b1fb994e237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.298437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa071d-3abf-4ff0-a8aa-8b1fb994e237 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.298764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa071d-3abf-4ff0-a8aa-8b1fb994e237 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.299044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fa071d-3abf-4ff0-a8aa-8b1fb994e237 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.299880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa071d-3abf-4ff0-a8aa-8b1fb994e237 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:10.300189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa071d-3abf-4ff0-a8aa-8b1fb994e237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.301984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fa071d-3abf-4ff0-a8aa-8b1fb994e237 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.315759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f55fb-9837-4157-ba33-a0ca9e96ada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.316421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f55fb-9837-4157-ba33-a0ca9e96ada5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.317237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f55fb-9837-4157-ba33-a0ca9e96ada5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.317514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f55fb-9837-4157-ba33-a0ca9e96ada5 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.318340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 415/834] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:10.318453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f55fb-9837-4157-ba33-a0ca9e96ada5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:10.318698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f55fb-9837-4157-ba33-a0ca9e96ada5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.319655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7da3db-fd74-429d-a561-94f8cd31a330 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.319971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7da3db-fd74-429d-a561-94f8cd31a330 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.320142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f55fb-9837-4157-ba33-a0ca9e96ada5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.320295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7da3db-fd74-429d-a561-94f8cd31a330 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.325739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 416/835] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:10.325739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7da3db-fd74-429d-a561-94f8cd31a330 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.342754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 420/836] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:10.344103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79413f18-58f0-42c4-bdb3-d43c037e1387 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.344465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79413f18-58f0-42c4-bdb3-d43c037e1387 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.344746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79413f18-58f0-42c4-bdb3-d43c037e1387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.361289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-79413f18-58f0-42c4-bdb3-d43c037e1387 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:10.362527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-79413f18-58f0-42c4-bdb3-d43c037e1387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:10.439893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 421/837] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:10.440576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79413f18-58f0-42c4-bdb3-d43c037e1387 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.451299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7da3db-fd74-429d-a561-94f8cd31a330 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.460219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:10.473994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:10.500053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.501067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.501180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.501394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.513654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.515266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:10.534289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:10.535396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:10.537337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:10.537337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:10.537337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:10.537337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:10.537337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 417/838] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:10.537337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.560790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79413f18-58f0-42c4-bdb3-d43c037e1387 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.601120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.602242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.602594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.602875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/bf00375b0f334f7bada8f21ea3766ac6/roles/763f046b87da4abbaff504f77c55288c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.621350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=bf00375b0f334f7bada8f21ea3766ac6, role_id=763f046b87da4abbaff504f77c55288c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.622954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.638370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 422/839] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:10 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/bf00375b0f334f7bada8f21ea3766ac6/roles/763f046b87da4abbaff504f77c55288c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:10.639563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.667186 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac3bc1-9d7d-4241-82b3-6fbd20f34c1b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.724744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.726399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.727294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.728647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.730103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.733289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.742162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:10.750815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:10.752087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:10.753118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 418/840] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:10.758626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.793410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cff463-56c3-497e-b0ee-cb12f841fc61 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.809231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:10.841073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.842262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.842716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.843269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.844107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:10.844422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.845945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.858993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 423/841] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:10.865427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:10.922784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e99824-d605-44c7-bff9-192dc73502c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.972326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:10.973059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:10.973589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:10.973914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:10.975054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:10.975510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:10.977282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:10.980421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0ac61-79d3-47fb-bd87-9ac3f2135775 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:10.989872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 419/842] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:10.992512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:11.011790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:11.021711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:11.022595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.022889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.023225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.032046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:11.033575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:11.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:11.054223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 16:00:11.055147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:11.063053 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 424/843] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:11.064726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:11.123391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:11.144030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109524cc-ce03-4735-91b1-e5d08fde3be2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:11.184633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:11.185356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.185682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.186089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.186786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:11.187103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:11.188613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:11.200559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 420/844] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:11.216793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e22ef-b543-4e70-9eee-19ad79b25054 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:11.259082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395cf38-b2dc-4c01-a7cd-de5b9cccc568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:11.259814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395cf38-b2dc-4c01-a7cd-de5b9cccc568 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.260215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395cf38-b2dc-4c01-a7cd-de5b9cccc568 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.260555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395cf38-b2dc-4c01-a7cd-de5b9cccc568 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.261533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395cf38-b2dc-4c01-a7cd-de5b9cccc568 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:11.261884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395cf38-b2dc-4c01-a7cd-de5b9cccc568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:11.263491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395cf38-b2dc-4c01-a7cd-de5b9cccc568 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:11.288069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 425/845] 10.222.0.88 () {56 vars in 1135 bytes} [Thu Jun 16 16:00:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:11.298718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:11.414264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491a73b-efc4-4ae7-ae35-5ae88948d869 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:11.454523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:11.455649 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.456153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.456869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.485923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:11.531284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:11.533129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:11.626872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41cd8-f6f2-4e04-b9db-60a58c6b5bbc None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.627418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41cd8-f6f2-4e04-b9db-60a58c6b5bbc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.627418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41cd8-f6f2-4e04-b9db-60a58c6b5bbc None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.629042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 426/846] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:11.634459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b1616-0ea0-429b-bacc-0d9346033d4b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.634751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b1616-0ea0-429b-bacc-0d9346033d4b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.635035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b1616-0ea0-429b-bacc-0d9346033d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.647779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 421/847] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:11.652238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-692b1616-0ea0-429b-bacc-0d9346033d4b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:11.653459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-692b1616-0ea0-429b-bacc-0d9346033d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:11.654437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:11.710450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:11.729809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 427/848] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:11.738083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:11.752699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b1616-0ea0-429b-bacc-0d9346033d4b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:11.786020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89998d4f-d5a9-49b4-959c-02df451b6734 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:11.824831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:11.825631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.825945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.826313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.837399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:11.838971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:11.858497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:11.859633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:11.860618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:11.860618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:11.860618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:11.860618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:11.862733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b1616-0ea0-429b-bacc-0d9346033d4b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:11.864463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 422/849] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:11.866596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:11.905670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:11.907015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:11.907467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:11.907898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:11.908803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:11.911816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:11.926001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:11.927330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:11.928372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:11.930607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 428/850] 10.222.0.88 () {56 vars in 1132 bytes} [Thu Jun 16 16:00:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:11.936347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:11.984147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:11.999153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751b07-8125-419d-a554-a9b70aefcf2a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.010552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:12.040329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.041159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.041550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.041980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.042981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:12.043412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.045050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.050228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f4f0-7f80-4672-8684-7b58daf46f49 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.058311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 423/851] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:12.064662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.087602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.088532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.088980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.089555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.090442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:12.090921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.092749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.098265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 429/852] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:12.103761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.212260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bbb87-1e6c-49d7-b78d-c1ffe8191853 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.244087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1ce0-8d67-4848-a9c7-2ab8ed8bca02 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.246784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:12.258115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.259221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.260041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.260476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.270401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.272600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.288102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.289055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.289487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.289963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.290907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:12.291107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:12.292137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:12.293201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:12.293743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.295438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 424/853] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:12.300877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.302739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.308203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 430/854] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:12.346709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:12.435409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe1426-f58a-4da0-ac2a-b055f39db548 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.480562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b67d8-a78d-4e18-be0e-b6122a772ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.481505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b67d8-a78d-4e18-be0e-b6122a772ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.482027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b67d8-a78d-4e18-be0e-b6122a772ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.482473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b67d8-a78d-4e18-be0e-b6122a772ba1 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.483426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b67d8-a78d-4e18-be0e-b6122a772ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:12.483909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b67d8-a78d-4e18-be0e-b6122a772ba1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.485596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b67d8-a78d-4e18-be0e-b6122a772ba1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.498982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 425/855] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:12.505217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:12.615348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a87c3d-1fec-4974-a131-52273b8fe134 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:12.652064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4c8f3-20bd-4cce-a081-bea836e59d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:12.653407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4c8f3-20bd-4cce-a081-bea836e59d3f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:12.653919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4c8f3-20bd-4cce-a081-bea836e59d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:12.654349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4c8f3-20bd-4cce-a081-bea836e59d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:12.655342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4c8f3-20bd-4cce-a081-bea836e59d3f None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=9b4b1f0775ba4875a550ea986ed483e7) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:12.655907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4c8f3-20bd-4cce-a081-bea836e59d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:12.657502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4c8f3-20bd-4cce-a081-bea836e59d3f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:12.665134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 431/856] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:12 2022] GET /identity/v3/role_assignments?user.id=9b4b1f0775ba4875a550ea986ed483e7&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:13.018063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e80041-c578-44f1-9275-e645ada7df1b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.018485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e80041-c578-44f1-9275-e645ada7df1b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.018682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e80041-c578-44f1-9275-e645ada7df1b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.020809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 426/857] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:13.026422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d7684-8144-430e-bafe-ee67c7bd78db None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.026717 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d7684-8144-430e-bafe-ee67c7bd78db None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.027005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d7684-8144-430e-bafe-ee67c7bd78db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.044492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-887d7684-8144-430e-bafe-ee67c7bd78db None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:13.045681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-887d7684-8144-430e-bafe-ee67c7bd78db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:13.076581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:13.121933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 432/858] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:13.146569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e80041-c578-44f1-9275-e645ada7df1b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:13.240390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:13.255103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e80041-c578-44f1-9275-e645ada7df1b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:13.292581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:13.293371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.293653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.294014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.306103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:13.314280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:13.315242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:13.316089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 427/859] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 16:00:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:13.322013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d7684-8144-430e-bafe-ee67c7bd78db None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:13.428055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d7684-8144-430e-bafe-ee67c7bd78db None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:13.465951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd774e4-8d4e-4853-ae21-024ecb4521e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:13.466901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd774e4-8d4e-4853-ae21-024ecb4521e9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.467186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd774e4-8d4e-4853-ae21-024ecb4521e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.467464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd774e4-8d4e-4853-ae21-024ecb4521e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.468214 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd774e4-8d4e-4853-ae21-024ecb4521e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:13.468698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd774e4-8d4e-4853-ae21-024ecb4521e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:13.470131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd774e4-8d4e-4853-ae21-024ecb4521e9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:13.479627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 433/860] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 16:00:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:13.485499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:13.562423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b049-ab8a-4e9b-b3a0-3c80f8c3c161 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.563098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b049-ab8a-4e9b-b3a0-3c80f8c3c161 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.563650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883b049-ab8a-4e9b-b3a0-3c80f8c3c161 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.565552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 434/861] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:13.575527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2799-cd9c-46ad-bb3d-c139dbe6d3ac None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.576133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2799-cd9c-46ad-bb3d-c139dbe6d3ac None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.576650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2799-cd9c-46ad-bb3d-c139dbe6d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.594343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-25ef2799-cd9c-46ad-bb3d-c139dbe6d3ac None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:13.595937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-25ef2799-cd9c-46ad-bb3d-c139dbe6d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:13.597494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c5354-5b2f-4ac8-98d8-45a8b8b5cce3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:13.618091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:13.639508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:13.640960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.641463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.642008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.654332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:13.656191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:13.666306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:13.678085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:13.678997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:13.679894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:13.679894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:13.679894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:13.679894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:13.686340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 428/862] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:13.687790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 435/863] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:13.688559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:13.712067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2799-cd9c-46ad-bb3d-c139dbe6d3ac None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:13.802776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb83144-0d4c-472d-a4c9-901fa7d843a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:13.818132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2799-cd9c-46ad-bb3d-c139dbe6d3ac None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:13.840107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:13.841161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.842042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.842555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.843476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:13.843988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:13.845593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:13.855801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:13.856473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:13.856762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:13.857042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:13.857790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:13.859170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:13.865031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 429/864] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:13.866528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:13.885751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 436/865] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:13 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:00:14.062434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:14.109246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d2936-58d7-46ad-ba4a-5b25db929bb6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:14.154482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:14.155334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.155616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.155889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.165051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:14.167048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:14.183796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:14.184893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:14.185906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:14.188742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 430/866] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:00:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:14.201753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:14.212056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:14.305397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394710-c15a-4bed-bd7d-43f85c8095b9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:14.345026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f64c2-e4ad-4647-b10d-c803bbe40856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:14.346545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f64c2-e4ad-4647-b10d-c803bbe40856 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.347035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f64c2-e4ad-4647-b10d-c803bbe40856 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.347599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f64c2-e4ad-4647-b10d-c803bbe40856 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.349146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f64c2-e4ad-4647-b10d-c803bbe40856 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:14.349669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f64c2-e4ad-4647-b10d-c803bbe40856 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:14.351544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f64c2-e4ad-4647-b10d-c803bbe40856 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:14.371240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 437/867] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 16:00:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:14.376921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:14.489680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e170803-a454-432d-b8d6-c81ec49a25be None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:14.509532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:14.530003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:14.530667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.530967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.531318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.560012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea730-e08c-4b3b-a5e9-3cf72af384ce None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.560325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea730-e08c-4b3b-a5e9-3cf72af384ce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.560573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea730-e08c-4b3b-a5e9-3cf72af384ce None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.562571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 438/868] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:14.567843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982c598-9b3e-4d41-9bd8-8a39be41fbf2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.568133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982c598-9b3e-4d41-9bd8-8a39be41fbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.568416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982c598-9b3e-4d41-9bd8-8a39be41fbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.586623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e982c598-9b3e-4d41-9bd8-8a39be41fbf2 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:14.587805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e982c598-9b3e-4d41-9bd8-8a39be41fbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:14.605817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:14.608112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:14.679072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 439/869] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:14.700944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982c598-9b3e-4d41-9bd8-8a39be41fbf2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:14.703302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:14.742383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 431/870] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:14 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:14.747806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:14.770806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:14.822503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982c598-9b3e-4d41-9bd8-8a39be41fbf2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:14.864818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9aac8-94ef-4c11-9b08-cbbf5491dc5d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:14.866715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:14.868270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.868642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.868999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.869906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:14.871300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:14.890649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 440/871] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:14 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:14.917573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40772217-61d9-4537-aacf-160609218fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:14.918681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40772217-61d9-4537-aacf-160609218fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:14.918979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40772217-61d9-4537-aacf-160609218fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:14.919252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40772217-61d9-4537-aacf-160609218fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:14.932973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40772217-61d9-4537-aacf-160609218fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:14.934523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40772217-61d9-4537-aacf-160609218fb5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-40772217-61d9-4537-aacf-160609218fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:14.956100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:14.957070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:14.958561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:14.958561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:14.958561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:14.958561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:14.958803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 432/872] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:14.964170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.038470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.084570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ffc69-439b-4ebf-a6e5-83b6945c7cb5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.103822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6aa14-9fd5-4449-90c1-5614211dd600 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.104127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6aa14-9fd5-4449-90c1-5614211dd600 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.104393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6aa14-9fd5-4449-90c1-5614211dd600 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.110003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 433/873] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:15.112584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcc23a-001a-4ac8-bb93-07727e8bdcb7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.113127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcc23a-001a-4ac8-bb93-07727e8bdcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.113407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcc23a-001a-4ac8-bb93-07727e8bdcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.128279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.128982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.129817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.130104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.133688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:15.133688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.133688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.135308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-53fcc23a-001a-4ac8-bb93-07727e8bdcb7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:15.136372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53fcc23a-001a-4ac8-bb93-07727e8bdcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:15.153080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 441/874] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:15.161883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.213958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.225842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 434/875] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:15.251973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcc23a-001a-4ac8-bb93-07727e8bdcb7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.288464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e50515-0bf6-4854-b81f-8ae51bc61650 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.317104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.331532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.333115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.333402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.333676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.342289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.344430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:15.360468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:15.361483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:15.362404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:15.367889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 442/876] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:15.369237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.380754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcc23a-001a-4ac8-bb93-07727e8bdcb7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.424295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.425565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.426047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.427029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.428215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.430204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:15.446028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:15.447064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:15.447900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:15.450968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 435/877] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:15.456933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.492994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d453706-ebbf-4d79-a9ce-8a168eb9de4e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.530901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.535550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.536526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.537134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.537608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.539130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:15.539585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.541192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.556675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 443/878] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:15.563034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.586585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c974ce-f71c-4b66-b795-facbe5676c58 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.623810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.635131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.636306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.636869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.637480 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.638682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:15.639203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.641189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.652881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 436/879] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:15.654553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:15.685583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7cd4a9-0503-4db9-82c3-35736d70cb10 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.722528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd647731-1d1f-4b79-a8c8-43a24684bfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.723907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd647731-1d1f-4b79-a8c8-43a24684bfcb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.724149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd647731-1d1f-4b79-a8c8-43a24684bfcb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.724379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd647731-1d1f-4b79-a8c8-43a24684bfcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.725207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd647731-1d1f-4b79-a8c8-43a24684bfcb None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=bf00375b0f334f7bada8f21ea3766ac6) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:15.725519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd647731-1d1f-4b79-a8c8-43a24684bfcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.726986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd647731-1d1f-4b79-a8c8-43a24684bfcb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.737895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 444/880] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/role_assignments?user.id=bf00375b0f334f7bada8f21ea3766ac6&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:15.792112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ed287-d2a2-4583-869f-132dbe6048d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:15.822731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:15.839574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf3a53-1e99-4104-b131-cf984e604f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:15.840776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf3a53-1e99-4104-b131-cf984e604f11 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:15.841643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf3a53-1e99-4104-b131-cf984e604f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:15.842416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf3a53-1e99-4104-b131-cf984e604f11 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:15.844183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf3a53-1e99-4104-b131-cf984e604f11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:15.844980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf3a53-1e99-4104-b131-cf984e604f11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:15.847035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf3a53-1e99-4104-b131-cf984e604f11 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:15.861817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 437/881] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:16.251170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102213-7b4d-4448-a780-72c3baaf0382 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:16.251499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102213-7b4d-4448-a780-72c3baaf0382 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:16.251752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c102213-7b4d-4448-a780-72c3baaf0382 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:16.253403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 445/882] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:16.258770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de070a0b-8661-4df5-9d32-9bae4a96857b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:16.259061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de070a0b-8661-4df5-9d32-9bae4a96857b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:16.259344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de070a0b-8661-4df5-9d32-9bae4a96857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:16.277851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-de070a0b-8661-4df5-9d32-9bae4a96857b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:16.279031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-de070a0b-8661-4df5-9d32-9bae4a96857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:16.359448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 438/883] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:16.383168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c102213-7b4d-4448-a780-72c3baaf0382 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:16.503947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c102213-7b4d-4448-a780-72c3baaf0382 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:16.513431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:16.548132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:16.549340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:16.550011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:16.550619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:16.551627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:16.553329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:16.568318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:16.569574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:16.570511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:16.573842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 446/884] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:16.579827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de070a0b-8661-4df5-9d32-9bae4a96857b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:16.699466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de070a0b-8661-4df5-9d32-9bae4a96857b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:16.720894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:16.743945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:16.744616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:16.744955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:16.745301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:16.746151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:16.746458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:16.748009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:16.754400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 439/885] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:16.760035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:16.875276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016548cd-126c-4588-8e92-5a57e19c5f1d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:16.915104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:16.915913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:16.916191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:16.916518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:16.917383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:16.917749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:16.919313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:16.930140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 447/886] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:16.932454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:17.052185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dcd9b-0e73-4ff4-8447-53d1addfef8b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:17.099538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead23925-a864-4a9b-bd71-7276e8aeb303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:17.100458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23925-a864-4a9b-bd71-7276e8aeb303 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.100816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23925-a864-4a9b-bd71-7276e8aeb303 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.101156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead23925-a864-4a9b-bd71-7276e8aeb303 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.102221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead23925-a864-4a9b-bd71-7276e8aeb303 None admin] RBAC: Adding query filter params (interface=public, service_id=480166c32f414d238ee64bdb640dcf34, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:17.102624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead23925-a864-4a9b-bd71-7276e8aeb303 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:17.104285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead23925-a864-4a9b-bd71-7276e8aeb303 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:17.116503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 440/887] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:00:16 2022] GET /identity/v3/endpoints?service_id=480166c32f414d238ee64bdb640dcf34&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:17.117968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:17.134262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64ff72-961c-427d-a9fb-28dc1c76f520 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.134562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64ff72-961c-427d-a9fb-28dc1c76f520 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.134821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a64ff72-961c-427d-a9fb-28dc1c76f520 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.136632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 441/888] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:17.142936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a00501-f489-4acf-a7a2-7541bf7a3754 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.143224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a00501-f489-4acf-a7a2-7541bf7a3754 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.143506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a00501-f489-4acf-a7a2-7541bf7a3754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.160288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36a00501-f489-4acf-a7a2-7541bf7a3754 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:17.162315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36a00501-f489-4acf-a7a2-7541bf7a3754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:17.169515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:17.212421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:17.252686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ce28e-3489-4a49-a22c-504aeb642561 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:17.253257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 442/889] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:17.276602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a00501-f489-4acf-a7a2-7541bf7a3754 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:17.292637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:17.293367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.293780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.293975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.294773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:17.296199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:17.308069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 448/890] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:17 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:00:17.397471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a00501-f489-4acf-a7a2-7541bf7a3754 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:17.436316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:17.437555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.438081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.438560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.439427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:17.441023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:17.454672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:17.455698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:17.456552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:17.460193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 443/891] 10.222.0.88 () {56 vars in 1132 bytes} [Thu Jun 16 16:00:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:17.465898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:17.572521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:17.577388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e422482-f98e-466a-8343-8ebdd717bd58 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:17.615134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:17.615806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.616092 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.616371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.617076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:17.617388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:17.618815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:17.624196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 449/892] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:17.630060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:17.679711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:17.744680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d14a3e5-198c-44ed-84b5-36a44c894393 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:17.780956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:17.782027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.782568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.783048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.783958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:17.784466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:17.786166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:17.791828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 444/893] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:17.798686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:17.912670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827deda4-f54c-492f-bc66-5626cd371d83 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:17.951618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c332ec-11dd-45b0-9e0c-e061e692301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:17.952316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c332ec-11dd-45b0-9e0c-e061e692301a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:17.952604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c332ec-11dd-45b0-9e0c-e061e692301a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:17.952886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c332ec-11dd-45b0-9e0c-e061e692301a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:17.953781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c332ec-11dd-45b0-9e0c-e061e692301a None admin] RBAC: Adding query filter params (interface=public, service_id=cba2ac7c38354e5381c98246fc62d6a7, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:17.954146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c332ec-11dd-45b0-9e0c-e061e692301a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:17.955602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c332ec-11dd-45b0-9e0c-e061e692301a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:17.962322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 450/894] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:00:17 2022] GET /identity/v3/endpoints?service_id=cba2ac7c38354e5381c98246fc62d6a7&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:17.968167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.031446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b78b-7be8-4de1-acc3-78792ba8f368 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.032078 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b78b-7be8-4de1-acc3-78792ba8f368 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.032606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b78b-7be8-4de1-acc3-78792ba8f368 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.034628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 451/895] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:18.039979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5da1-6741-4b76-aab8-8937ac109af0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.040267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5da1-6741-4b76-aab8-8937ac109af0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.040568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5da1-6741-4b76-aab8-8937ac109af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.058974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e2bd5da1-6741-4b76-aab8-8937ac109af0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:18.060500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e2bd5da1-6741-4b76-aab8-8937ac109af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:18.085546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.089244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be9413-89d6-425a-b878-abbe27adeaf0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.092357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.131941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a330a-0492-4d8a-b116-5e1929278c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.132639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a330a-0492-4d8a-b116-5e1929278c6b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.132925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a330a-0492-4d8a-b116-5e1929278c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.133206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a330a-0492-4d8a-b116-5e1929278c6b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.133956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a330a-0492-4d8a-b116-5e1929278c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:18.135843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a330a-0492-4d8a-b116-5e1929278c6b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:18.142318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 445/896] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:17 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:00:18.160124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 452/897] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:18.181875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d44952-02e3-4626-882a-4a3fe2365c6d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.182413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d44952-02e3-4626-882a-4a3fe2365c6d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.182850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d44952-02e3-4626-882a-4a3fe2365c6d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.184454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 446/898] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:18.189765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd5da1-6741-4b76-aab8-8937ac109af0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.190294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61876d-fe10-4ad1-8631-86368094bc7b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.190685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61876d-fe10-4ad1-8631-86368094bc7b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.190973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61876d-fe10-4ad1-8631-86368094bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.213174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f61876d-fe10-4ad1-8631-86368094bc7b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:18.214408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f61876d-fe10-4ad1-8631-86368094bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:18.302173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 447/899] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:18.321780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61876d-fe10-4ad1-8631-86368094bc7b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.323167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd5da1-6741-4b76-aab8-8937ac109af0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.369869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.371135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.371424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.371704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.389835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:18.399349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:18.400300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:18.401151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 453/900] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 16:00:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:18.411924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.433746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.443290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.452199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61876d-fe10-4ad1-8631-86368094bc7b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.494252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.494958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.495240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.495518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.496248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:18.497631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:18.514408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 448/901] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:18 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:18.554622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0072b-f94c-46fd-bc6c-67598f67c457 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.606437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.607175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.607480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.607804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.608678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:18.609005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:18.610678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:18.622727 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 454/902] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 16:00:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:18.628756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.745168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740a515-a96a-4dd7-8d68-348a1b34c311 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.781836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.782683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.782971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.783242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.795208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:18.796661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:18.814433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:18.815376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:18.816366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:18.816366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:18.816366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:18.816366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:18.816366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 449/903] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:18.823669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:18.915117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:18.943472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd47499-902c-416d-9ee6-9765d79416bc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:18.980731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:18.981479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:18.981819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:18.982106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:18.982856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:18.983195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:18.984592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:18.998231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 455/904] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:19.004188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:19.016883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:19.118215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bd68d-2545-41b7-bb0f-7387d4ff9cd5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:19.155590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:19.156479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.156765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.157055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.165621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:19.167331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:19.183746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:19.184702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:19.185627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 450/905] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:00:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:19.191215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:19.295543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a718b-5f6b-4562-9b48-688e9daae6af None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:19.331239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273b5dd-b620-4abe-be1a-61a2f3f205ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:19.331928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273b5dd-b620-4abe-be1a-61a2f3f205ea None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.332212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273b5dd-b620-4abe-be1a-61a2f3f205ea None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.332554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273b5dd-b620-4abe-be1a-61a2f3f205ea None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.333411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273b5dd-b620-4abe-be1a-61a2f3f205ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:19.333739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273b5dd-b620-4abe-be1a-61a2f3f205ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:19.335236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273b5dd-b620-4abe-be1a-61a2f3f205ea None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:19.337231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:19.356685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 456/906] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 16:00:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:19.362321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:19.436759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:19.475074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a421ab-9dc1-4850-9f29-0889fe225655 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:19.512166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:19.512905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.513219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.513498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.587627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:19.589155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:19.670121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:19.689765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65241fc-e7e5-4284-8e71-1890d0142e06 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.690380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65241fc-e7e5-4284-8e71-1890d0142e06 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.690864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65241fc-e7e5-4284-8e71-1890d0142e06 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.692723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 457/907] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:19.698108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366a17-c69d-47bb-8283-429b31d7c0ad None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.698739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366a17-c69d-47bb-8283-429b31d7c0ad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.699225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53366a17-c69d-47bb-8283-429b31d7c0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.707245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 451/908] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:19 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:19.714043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:19.718897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-53366a17-c69d-47bb-8283-429b31d7c0ad None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:19.720327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-53366a17-c69d-47bb-8283-429b31d7c0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:19.801796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 458/909] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:19.815548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53366a17-c69d-47bb-8283-429b31d7c0ad None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:19.825808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84927676-6270-43aa-af07-328f615e1e3e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:19.872664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:19.873916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.874456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.874958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.887350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:19.889235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:19.909285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:19.910260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:19.911240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:19.911240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:19.911240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:19.911240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:19.913871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 452/910] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:19.919737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:19.937848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53366a17-c69d-47bb-8283-429b31d7c0ad None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:19.953043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:19.965603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:19.979983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:19.981186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:19.981701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:19.982228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:19.983172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:19.984945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:19.998726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:19.999762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 16:00:20.000686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:20.008273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 459/911] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:20.009447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.048807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e241a-f8a9-4952-8ae6-1a5e2af85085 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.087187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.088271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.088778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.089402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.090618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:20.091145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.092970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.106991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 453/912] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:20.114284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.143695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073a2d4-c6f3-4f75-ac0b-b92c527939ef None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.194104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.196239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.196882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.198107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.198999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:20.199317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.201659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.208300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 460/913] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:20.213817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.214478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:20.241596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b228b45-3dfa-4257-9c6c-6672bdb55067 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.272431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:20.288775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.290178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.290713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.291192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.302947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.305173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:20.325467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:20.326596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:20.327530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:20.334743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd9d33-4b9a-4e73-8441-d643ae143cec None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.335008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 454/914] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:20.336183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.373017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.373852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.374213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.374607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.375621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:20.376018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.377764 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.384894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 461/915] 10.222.0.88 () {56 vars in 1114 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:20.391276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.456414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda9c6-dcba-4d79-927c-322ad092dfef None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.500256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6992a-dd2b-49b2-af5b-b3ff171b9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.501116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6992a-dd2b-49b2-af5b-b3ff171b9240 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.501396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6992a-dd2b-49b2-af5b-b3ff171b9240 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.501672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6992a-dd2b-49b2-af5b-b3ff171b9240 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.502598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6992a-dd2b-49b2-af5b-b3ff171b9240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:20.502941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6992a-dd2b-49b2-af5b-b3ff171b9240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.504488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6992a-dd2b-49b2-af5b-b3ff171b9240 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.505778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395cfd3-1a3c-4313-9a2c-97789e54a88e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.514276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:20.522460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:20.524087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 455/916] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:20.525571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb432c16-0f0b-44f1-bfc8-a318be085378 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.525897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb432c16-0f0b-44f1-bfc8-a318be085378 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.526177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb432c16-0f0b-44f1-bfc8-a318be085378 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.529801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 456/917] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:20.529801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb432c16-0f0b-44f1-bfc8-a318be085378 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.551289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dacf09-76b5-4263-a7ec-4bd784dac3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.552255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dacf09-76b5-4263-a7ec-4bd784dac3f8 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.552544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dacf09-76b5-4263-a7ec-4bd784dac3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.552827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dacf09-76b5-4263-a7ec-4bd784dac3f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.553582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dacf09-76b5-4263-a7ec-4bd784dac3f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.555565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dacf09-76b5-4263-a7ec-4bd784dac3f8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.581762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 462/918] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:20.581762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c98e8-0e8e-4178-86aa-d86b7c1d3558 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.581762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c98e8-0e8e-4178-86aa-d86b7c1d3558 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.581762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c98e8-0e8e-4178-86aa-d86b7c1d3558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.606134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3f5c98e8-0e8e-4178-86aa-d86b7c1d3558 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:20.607795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3f5c98e8-0e8e-4178-86aa-d86b7c1d3558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:20.668955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb432c16-0f0b-44f1-bfc8-a318be085378 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.702510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 463/919] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:20.715933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726a0d2-8b6c-45ed-9506-dbf400d1b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.716871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726a0d2-8b6c-45ed-9506-dbf400d1b56f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.717169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726a0d2-8b6c-45ed-9506-dbf400d1b56f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.717454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726a0d2-8b6c-45ed-9506-dbf400d1b56f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.718371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726a0d2-8b6c-45ed-9506-dbf400d1b56f None admin] RBAC: Adding query filter params (role.id=66544664463c4fb9bf1c9d5bce6df901, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=bf00375b0f334f7bada8f21ea3766ac6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:20.718812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726a0d2-8b6c-45ed-9506-dbf400d1b56f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.720534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726a0d2-8b6c-45ed-9506-dbf400d1b56f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.728933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c98e8-0e8e-4178-86aa-d86b7c1d3558 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.730870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 457/920] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/role_assignments?user.id=bf00375b0f334f7bada8f21ea3766ac6&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=66544664463c4fb9bf1c9d5bce6df901 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:20.792326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f2c1-98b6-4c02-bb7c-8f377633d0b8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.792638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f2c1-98b6-4c02-bb7c-8f377633d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.792906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f2c1-98b6-4c02-bb7c-8f377633d0b8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.794987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 458/921] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:20.802199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936134cb-d4d0-4ea6-9990-33cfb7ec1eac None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.802641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936134cb-d4d0-4ea6-9990-33cfb7ec1eac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.802932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936134cb-d4d0-4ea6-9990-33cfb7ec1eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.824888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-936134cb-d4d0-4ea6-9990-33cfb7ec1eac None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:20.826371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-936134cb-d4d0-4ea6-9990-33cfb7ec1eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:20.875417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:20.879693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c98e8-0e8e-4178-86aa-d86b7c1d3558 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:20.910220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:20.927703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:20.928829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:20.929293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:20.929783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:20.930647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:20.931878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 459/922] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:20.932398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:20.947134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:20.948529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:20.949362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 464/923] 10.222.0.88 () {56 vars in 1132 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:20.954809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936134cb-d4d0-4ea6-9990-33cfb7ec1eac None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:20.961333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.076756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936134cb-d4d0-4ea6-9990-33cfb7ec1eac None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.081266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47131dc-af92-44f9-ace5-4d1f14b54160 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.118895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.119700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.119982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.120259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.120910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.122346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.125992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.127171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.127677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.128167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.129068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:21.130135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.131708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.137399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 460/924] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:21.147133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:21.147780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:21.148765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:21.149611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:21.153891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 465/925] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:21.155144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.186890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:21.228981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:21.266771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55b9b0-13ac-4894-8942-bdddf5686ba4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.278284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44052376-329e-4683-9fc4-a8bc9e5aa9ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.302935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.303614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.303899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.304179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.304892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:21.305200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.306609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.312040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 466/926] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:21.325741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.325926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.325926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.325926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.325926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.326241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:21.326241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.326241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.330848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 461/927] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:21.336764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.431874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:21.442407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f433a-a4bf-4881-8ef1-1515aa5c1162 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.460659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70320435-5ab2-4b2a-ae4a-5c021890f763 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.482118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.483065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.483547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.483970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.484428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:21.486297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:21.487032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.489927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.496134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 467/928] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:21.503983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.507365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.508617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.508617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.508617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.509295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.510719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.536262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 462/929] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:21.635300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cc5fe-c203-4f0e-bcf2-4d414aefde94 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.680564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ad984-946e-4ea4-bd38-35d7a14593e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.681564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad984-946e-4ea4-bd38-35d7a14593e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.682051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad984-946e-4ea4-bd38-35d7a14593e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.682536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ad984-946e-4ea4-bd38-35d7a14593e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.683517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad984-946e-4ea4-bd38-35d7a14593e0 None admin] RBAC: Adding query filter params (interface=public, service_id=8dfec4c870504fd1b8a47698c2a04d47, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:21.684014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad984-946e-4ea4-bd38-35d7a14593e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.685734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ad984-946e-4ea4-bd38-35d7a14593e0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.692874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 468/930] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:00:21 2022] GET /identity/v3/endpoints?service_id=8dfec4c870504fd1b8a47698c2a04d47&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:21.699615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:21.807858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a88e4b-73f1-4e6d-9197-241a10b64475 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:21.847030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0863d07-e4b8-4b3c-9e71-690613cfcea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:21.847789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0863d07-e4b8-4b3c-9e71-690613cfcea4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:21.848520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0863d07-e4b8-4b3c-9e71-690613cfcea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:21.848799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0863d07-e4b8-4b3c-9e71-690613cfcea4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:21.849492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0863d07-e4b8-4b3c-9e71-690613cfcea4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:21.851040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0863d07-e4b8-4b3c-9e71-690613cfcea4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:21.861876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 463/931] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:21 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:00:22.951892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c53bd2-48a4-4179-80a2-2d7c36f3bcea None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:22.952420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c53bd2-48a4-4179-80a2-2d7c36f3bcea None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:22.952530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c53bd2-48a4-4179-80a2-2d7c36f3bcea None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:22.953885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 469/932] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:22.960903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead22b5-fd78-4fce-b9d9-1af8a50920e8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:22.961645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead22b5-fd78-4fce-b9d9-1af8a50920e8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:22.961645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead22b5-fd78-4fce-b9d9-1af8a50920e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:22.980128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cead22b5-fd78-4fce-b9d9-1af8a50920e8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:22.981329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cead22b5-fd78-4fce-b9d9-1af8a50920e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:22.994665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e23b24-de8a-40d7-b924-087d06588856 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:22.994948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e23b24-de8a-40d7-b924-087d06588856 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:22.995237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e23b24-de8a-40d7-b924-087d06588856 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:22.996923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 470/933] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:23.002878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e6385-b684-4daa-a60c-3bbde6856777 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.003173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e6385-b684-4daa-a60c-3bbde6856777 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.003459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e6385-b684-4daa-a60c-3bbde6856777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.019690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:23.030292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-863e6385-b684-4daa-a60c-3bbde6856777 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:23.031510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-863e6385-b684-4daa-a60c-3bbde6856777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:23.053930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:23.078630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 464/934] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:23.122684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead22b5-fd78-4fce-b9d9-1af8a50920e8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:23.175294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 471/935] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:23.222922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e6385-b684-4daa-a60c-3bbde6856777 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:23.279783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead22b5-fd78-4fce-b9d9-1af8a50920e8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:23.321207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:23.322022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.325739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.325739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.325739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:23.325739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:23.327057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING keystone.server.flask.common [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 16:00:23.349743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 16:00:23.349743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 16:00:23.349743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [parameters: {'id': '4c4b76f8ca104c04b19574c2b4e4e198', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 16:00:23.349743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4c4b76f8ca104c04b19574c2b4e4e198.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4c4b76f8ca104c04b19574c2b4e4e198. Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:00:23.356846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 16:00:23.357679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() Jun 16 16:00:23.358677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 16:00:23.359666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 16:00:23.360562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:00:23.361374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': '4c4b76f8ca104c04b19574c2b4e4e198', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:23.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:23.365811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4c4b76f8ca104c04b19574c2b4e4e198. Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.367915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 465/936] 10.222.0.88 () {60 vars in 1140 bytes} [Thu Jun 16 16:00:23 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:23.371573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:23.406217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:23.461560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e6385-b684-4daa-a60c-3bbde6856777 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:23.502276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:23.503105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.503515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.503773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.510709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0aa84e-8a55-4dd2-9552-1f0bd989fc18 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:23.515261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:23.524374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:23.525363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:23.526256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:23.530124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 472/937] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:23.532085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:23.542357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:23.553229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:23.554479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.554977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.555453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.563017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:23.564958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:23.580158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:23.581096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:23.582039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:23.585119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 466/938] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:23.590845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:23.652549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85b08e-7b02-4924-a207-706a5131a90c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:23.681635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:23.693198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:23.693895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.694182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.694463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.695208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:23.695522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:23.697070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:23.700599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724921a-e8e0-48ae-a901-53b9e14fad87 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:23.711555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 473/939] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:23.712460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:23.739144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eefe56-87f4-4358-8ae9-b0288c28b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:23.740214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eefe56-87f4-4358-8ae9-b0288c28b04d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.740703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eefe56-87f4-4358-8ae9-b0288c28b04d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.741182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eefe56-87f4-4358-8ae9-b0288c28b04d None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.742182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eefe56-87f4-4358-8ae9-b0288c28b04d None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:23.742693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eefe56-87f4-4358-8ae9-b0288c28b04d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:23.744350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eefe56-87f4-4358-8ae9-b0288c28b04d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:23.768267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 467/940] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:23.811776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:23.826904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f1bca-cf1d-40de-a104-f786df76dda3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.827338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f1bca-cf1d-40de-a104-f786df76dda3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.827739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f1bca-cf1d-40de-a104-f786df76dda3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.829517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 468/941] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:23.835746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a072be-1b6d-4620-9119-ded9ddf0342c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.836185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a072be-1b6d-4620-9119-ded9ddf0342c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.836568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a072be-1b6d-4620-9119-ded9ddf0342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.837744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b626289-62a7-4f5a-a391-7d9dd0a37601 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:23.861038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-55a072be-1b6d-4620-9119-ded9ddf0342c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:23.863237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-55a072be-1b6d-4620-9119-ded9ddf0342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:23.876950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:23.878700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:23.878989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:23.879839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:23.898458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:23.900059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:23.919385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:23.920464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:23.921398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:23.921398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:23.921398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:23.921398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:23.921398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 474/942] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:23.930813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:23.962002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 469/943] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:23.985790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a072be-1b6d-4620-9119-ded9ddf0342c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.078797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743032c-ed68-483f-9fea-9f056cd176ef None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.087644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.106568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.120185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a072be-1b6d-4620-9119-ded9ddf0342c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.125402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e6471-4007-4334-b2ab-ea9d22fc625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.126108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e6471-4007-4334-b2ab-ea9d22fc625b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.126393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e6471-4007-4334-b2ab-ea9d22fc625b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.126675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e6471-4007-4334-b2ab-ea9d22fc625b None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.127398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e6471-4007-4334-b2ab-ea9d22fc625b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:24.127703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e6471-4007-4334-b2ab-ea9d22fc625b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.129101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e6471-4007-4334-b2ab-ea9d22fc625b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.143033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 475/944] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:24.144207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e82d-43ab-472c-a79d-336f4de050ee None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.144499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e82d-43ab-472c-a79d-336f4de050ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.144834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87e82d-43ab-472c-a79d-336f4de050ee None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.146227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 476/945] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:24.150853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87e82d-43ab-472c-a79d-336f4de050ee None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.170470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fe36a-d11e-4423-896a-c5f1a45cdba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.171298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fe36a-d11e-4423-896a-c5f1a45cdba9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.171857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fe36a-d11e-4423-896a-c5f1a45cdba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.171970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fe36a-d11e-4423-896a-c5f1a45cdba9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.172618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fe36a-d11e-4423-896a-c5f1a45cdba9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.174097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fe36a-d11e-4423-896a-c5f1a45cdba9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7d2fe36a-d11e-4423-896a-c5f1a45cdba9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:24.189299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:24.190403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:24.191278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 470/946] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:24.192130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da5d9-c077-46b3-ab67-15d1991126b4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.192830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da5d9-c077-46b3-ab67-15d1991126b4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.192830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da5d9-c077-46b3-ab67-15d1991126b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.211876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a53da5d9-c077-46b3-ab67-15d1991126b4 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:24.213821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a53da5d9-c077-46b3-ab67-15d1991126b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:24.272155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87e82d-43ab-472c-a79d-336f4de050ee None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.301746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 471/947] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:24.301746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53da5d9-c077-46b3-ab67-15d1991126b4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.319165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.320000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.320286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.320560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.328123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.329624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:24.344766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:24.346166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:24.347329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:24.350049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 477/948] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:24.351763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.384478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.402433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.426982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53da5d9-c077-46b3-ab67-15d1991126b4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.466159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.467085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.467371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.467649 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.468350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:24.468727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.470739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ea60b-2f1a-4f56-a8f0-9464868dd994 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.479251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.489574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 472/949] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:24.490444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.518076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.519137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.519594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.520096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.520982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.522563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:24.535112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:24.536850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:24.537918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:24.540069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 478/950] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:24.540994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.606236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b2221-19a7-4f65-9b87-056241d023ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.641596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.649609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.650447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.650819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.651142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.652066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:24.652468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.652724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cf8d5-9662-4de2-a9ed-b82c765f761a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.654175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.655471 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.677671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 473/951] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:24.678712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.698878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.699835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.700270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.700724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.701587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:24.702166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.703668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.708986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 479/952] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:24.718947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.809361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e46865-4778-4ec8-b189-7e9e110fc808 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.860153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.861419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.861740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.862026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.863057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:24.863377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.866171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.874652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e469fa7e-3892-48ed-a396-9c5f82b1474b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:24.877079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 474/953] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:24.879752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:24.934182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:24.934991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:24.935281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:24.935588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:24.950565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:24.952477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:24.953634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:24.975128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:24.976151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:24.977127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:24.977127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:24.977127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:24.977127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:24.979594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 480/954] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:24.985477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:25.007267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb95e2-7c19-4ce0-b5a6-961f7e5b8d80 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:25.017268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:25.053036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:25.054112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:25.054612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:25.055103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:25.056027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:25.056575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:25.058308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:25.064467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 475/955] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:25.071816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:25.129188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43495d-6fc0-46ae-a7ab-953b424a26cf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:25.170958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:25.171712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:25.172141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:25.172441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:25.173251 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:25.173880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:25.175454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:25.183270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8336c7-31b7-46a9-afb9-dda82738a533 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:25.190467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 481/956] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:25.196132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:25.203252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:25.224858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:25.226412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:25.227119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:25.227818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:25.229465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:25.231156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:25.258110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 476/957] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:00:25.292762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:25.315856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc988941-3122-4d89-bd8d-6883052e993f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:25.363736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:25.364587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:25.364974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:25.365256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:25.373102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:25.374594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:25.390676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:25.391599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:25.392497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:25.395983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 482/958] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:25.398609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:25.509116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961807a2-aa30-4579-b632-74f5554854b6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:25.544728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb69d2a-0285-46fe-9215-2d251cb1eb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:25.545411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb69d2a-0285-46fe-9215-2d251cb1eb39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:25.545701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb69d2a-0285-46fe-9215-2d251cb1eb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:25.546013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb69d2a-0285-46fe-9215-2d251cb1eb39 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:25.546808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb69d2a-0285-46fe-9215-2d251cb1eb39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:25.547124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb69d2a-0285-46fe-9215-2d251cb1eb39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:25.548634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb69d2a-0285-46fe-9215-2d251cb1eb39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:25.565838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 477/959] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:25.568495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:25.678083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61abd76-2c4d-437c-afe6-07c2b2d77f99 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:25.713612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4d624-584f-4029-8dc0-229d9111d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:25.714543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4d624-584f-4029-8dc0-229d9111d78f None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:25.715140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4d624-584f-4029-8dc0-229d9111d78f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:25.715420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4d624-584f-4029-8dc0-229d9111d78f None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/bf00375b0f334f7bada8f21ea3766ac6/roles/66544664463c4fb9bf1c9d5bce6df901` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:25.717997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4d624-584f-4029-8dc0-229d9111d78f None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=bf00375b0f334f7bada8f21ea3766ac6, role_id=66544664463c4fb9bf1c9d5bce6df901)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:25.719451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4d624-584f-4029-8dc0-229d9111d78f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:25.722575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:25.741902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 483/960] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:25 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/bf00375b0f334f7bada8f21ea3766ac6/roles/66544664463c4fb9bf1c9d5bce6df901 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:26.156837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86371f-a332-4fee-ba47-82b41448265a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.157830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86371f-a332-4fee-ba47-82b41448265a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.158340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86371f-a332-4fee-ba47-82b41448265a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.160199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 478/961] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:26.165704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50247cd-6f31-400c-9b0b-43f3e1bd11f3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.166542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50247cd-6f31-400c-9b0b-43f3e1bd11f3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.167063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50247cd-6f31-400c-9b0b-43f3e1bd11f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.184606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f50247cd-6f31-400c-9b0b-43f3e1bd11f3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:26.186163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f50247cd-6f31-400c-9b0b-43f3e1bd11f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:26.266049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 484/962] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:26.292182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86371f-a332-4fee-ba47-82b41448265a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:26.336024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:26.414587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86371f-a332-4fee-ba47-82b41448265a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:26.452244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:26.453059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.453544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.454139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.454862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:26.456470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:26.473402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 16:00:26.473402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 16:00:26.473402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [parameters: {'user_id': '285367f344734188a897fb81f24489e3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 16:00:26.473402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jun 16 16:00:26.483709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:00:26.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 16:00:26.485745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 16:00:26.486687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 16:00:26.487844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 16:00:26.489495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'user_id': '285367f344734188a897fb81f24489e3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:26.496279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 16:00:26.498179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 16:00:26.500236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 16:00:26.500236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 16:00:26.500236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 16:00:26.500236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 16:00:26.500236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 16:00:26.500236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:26.500236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 479/963] 10.222.0.88 () {60 vars in 1131 bytes} [Thu Jun 16 16:00:26 2022] POST /identity/v3/users => generated 160 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:26.505762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50247cd-6f31-400c-9b0b-43f3e1bd11f3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:26.614727 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50247cd-6f31-400c-9b0b-43f3e1bd11f3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:26.645705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:26.654793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd6cd7-2335-4144-843b-440d34b7d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:26.655606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd6cd7-2335-4144-843b-440d34b7d1be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.655896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd6cd7-2335-4144-843b-440d34b7d1be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.656245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd6cd7-2335-4144-843b-440d34b7d1be None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.664041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd6cd7-2335-4144-843b-440d34b7d1be None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:26.665548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd6cd7-2335-4144-843b-440d34b7d1be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-93bd6cd7-2335-4144-843b-440d34b7d1be None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:26.682149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:26.684099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:26.685057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 485/964] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:26.692215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:26.794449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:26.800975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b9af4-844a-46b2-ab7b-a2569349d17d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:26.837041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57960e-95fd-46aa-b152-86feee49bb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:26.837730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57960e-95fd-46aa-b152-86feee49bb80 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.838030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57960e-95fd-46aa-b152-86feee49bb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.838310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57960e-95fd-46aa-b152-86feee49bb80 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.839266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57960e-95fd-46aa-b152-86feee49bb80 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:26.839583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57960e-95fd-46aa-b152-86feee49bb80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:26.841218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c57960e-95fd-46aa-b152-86feee49bb80 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:26.886495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 480/965] 10.222.0.88 () {56 vars in 1101 bytes} [Thu Jun 16 16:00:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:26.969399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a7f0b-b006-4b7a-b2a6-993beeecd0d2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.969588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a7f0b-b006-4b7a-b2a6-993beeecd0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.969904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a7f0b-b006-4b7a-b2a6-993beeecd0d2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.971605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 486/966] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:26.978055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1ee92-b468-4765-9784-7aaefb821219 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:26.978347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1ee92-b468-4765-9784-7aaefb821219 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:26.978620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1ee92-b468-4765-9784-7aaefb821219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:26.997547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-34e1ee92-b468-4765-9784-7aaefb821219 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:26.999172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34e1ee92-b468-4765-9784-7aaefb821219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:27.081223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 481/967] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:27.104509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a7f0b-b006-4b7a-b2a6-993beeecd0d2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:27.220698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a7f0b-b006-4b7a-b2a6-993beeecd0d2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:27.263983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1924db-a729-4a28-b4ff-e69d9f22a7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:27.264652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1924db-a729-4a28-b4ff-e69d9f22a7f4 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:27.264939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1924db-a729-4a28-b4ff-e69d9f22a7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:27.265302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1924db-a729-4a28-b4ff-e69d9f22a7f4 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:27.266075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1924db-a729-4a28-b4ff-e69d9f22a7f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:27.267443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1924db-a729-4a28-b4ff-e69d9f22a7f4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:27.282281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 487/968] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:27 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:27.651421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ea23e-4e05-433b-b778-3437dd849e01 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:27.651943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ea23e-4e05-433b-b778-3437dd849e01 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:27.651943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ea23e-4e05-433b-b778-3437dd849e01 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:27.653439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 482/969] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:27.658556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04ea88-52e6-412f-b791-6d69488b85ce None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:27.658845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04ea88-52e6-412f-b791-6d69488b85ce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:27.659127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04ea88-52e6-412f-b791-6d69488b85ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:27.676362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9f04ea88-52e6-412f-b791-6d69488b85ce None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:27.677537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f04ea88-52e6-412f-b791-6d69488b85ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:27.720215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:27.751590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 488/970] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:27.774764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ea23e-4e05-433b-b778-3437dd849e01 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:27.862039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:27.881016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ea23e-4e05-433b-b778-3437dd849e01 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:27.915399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1986244-690d-4c6f-9bf7-5aa30fb33f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:27.916119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1986244-690d-4c6f-9bf7-5aa30fb33f4e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:27.916400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1986244-690d-4c6f-9bf7-5aa30fb33f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:27.916675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1986244-690d-4c6f-9bf7-5aa30fb33f4e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:27.917327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1986244-690d-4c6f-9bf7-5aa30fb33f4e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:27.918718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1986244-690d-4c6f-9bf7-5aa30fb33f4e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e1986244-690d-4c6f-9bf7-5aa30fb33f4e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:27.930824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:27.931926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:27.933047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:27.937021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 483/971] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:27.937870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04ea88-52e6-412f-b791-6d69488b85ce None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.044957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04ea88-52e6-412f-b791-6d69488b85ce None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.047861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd1973-be04-4fcd-b232-32821cb82b4d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.048157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd1973-be04-4fcd-b232-32821cb82b4d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.048446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd1973-be04-4fcd-b232-32821cb82b4d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.049998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 484/972] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:28.054844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aac26b-51dd-416e-9c69-d0505515b576 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.055138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aac26b-51dd-416e-9c69-d0505515b576 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.055423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aac26b-51dd-416e-9c69-d0505515b576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.071964 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-06aac26b-51dd-416e-9c69-d0505515b576 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:28.073145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-06aac26b-51dd-416e-9c69-d0505515b576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:28.081080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.081774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.082063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.082875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.083586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:28.083896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:28.085267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:28.090550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 489/973] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:28.096324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.134737 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:28.159227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 485/974] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:28.183485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aac26b-51dd-416e-9c69-d0505515b576 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.211489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5187df8-762b-4865-badf-5ab7011f8ce9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.249133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.249910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.250181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.250488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.251269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:28.251608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:28.253130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:28.259688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 490/975] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:28.265496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.283277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:28.313153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aac26b-51dd-416e-9c69-d0505515b576 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.354175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.355119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.355445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.355738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.366979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:28.375662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:28.376609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:28.377453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 486/976] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:28.382683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:28.383120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.386583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ec36-1801-49f1-8d7b-b757e689e129 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.428795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.429487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.430226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.430514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.431330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] RBAC: Adding query filter params (interface=admin, service_id=8dfec4c870504fd1b8a47698c2a04d47, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:28.432012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:28.433812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:28.445754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 491/977] 10.222.0.88 () {56 vars in 1249 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/endpoints?service_id=8dfec4c870504fd1b8a47698c2a04d47&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:28.448135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.495001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cdb34-20de-4926-aec7-c8cba4e4ff57 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.536594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.537686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.538221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.538709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.539744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:28.540123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:28.541579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:28.547822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:28.556970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 487/978] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:28.557893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.569250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdf535-11d8-4e08-ad3d-8edd2ebe1018 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.607347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.608018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.608299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.608578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.609358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:28.610763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:28.617673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 492/979] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:00:28.681926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a980e-053d-427e-b26e-e58a070eb7f1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.719933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.720665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.720949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.721225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.732950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:28.734574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:28.753318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:28.754503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:28.755419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:28.755419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:28.755419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:28.755419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:28.755419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 488/980] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:28.760665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:28.795399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:28.880695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fe812-b5ad-4c12-9fb7-d51b5f6aeda8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:28.917380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:28.918122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:28.918402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:28.918674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:28.919387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:28.919688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:28.921041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:28.934266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 493/981] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:28.940126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.001408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.052937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d93d1-42a5-4efa-bb7c-520aae5dce8b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.088652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.089534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.089886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.090343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.100299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.102580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:29.119448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:29.120472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:29.121446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 489/982] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:00:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:29.126629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.224120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.237157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60121-22ca-4ca1-8664-72404f18829b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.274405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed21d6f-33cf-42b5-94b8-68e2c45770a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.275201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed21d6f-33cf-42b5-94b8-68e2c45770a9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.275549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed21d6f-33cf-42b5-94b8-68e2c45770a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.275866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed21d6f-33cf-42b5-94b8-68e2c45770a9 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.276826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed21d6f-33cf-42b5-94b8-68e2c45770a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:29.277134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed21d6f-33cf-42b5-94b8-68e2c45770a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.278923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed21d6f-33cf-42b5-94b8-68e2c45770a9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.300195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 494/983] 10.222.0.88 () {56 vars in 1149 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:29.305588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.415254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237c8b7-c63a-458c-a5d6-55e9fcc738e1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.430959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.455631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.456284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.456563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.456838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.464755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0995c-76b0-4f44-8cbb-a79a3a3d6d38 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.465038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0995c-76b0-4f44-8cbb-a79a3a3d6d38 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.465315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0995c-76b0-4f44-8cbb-a79a3a3d6d38 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.467216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 495/984] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:29.472246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030cc5b-b3bd-4707-bf76-8bdbaeefcc18 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.472539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030cc5b-b3bd-4707-bf76-8bdbaeefcc18 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.472824 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030cc5b-b3bd-4707-bf76-8bdbaeefcc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.489472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d030cc5b-b3bd-4707-bf76-8bdbaeefcc18 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:29.490683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d030cc5b-b3bd-4707-bf76-8bdbaeefcc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:29.534349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.535785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.558354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 496/985] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:29.580796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030cc5b-b3bd-4707-bf76-8bdbaeefcc18 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.641836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 490/986] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:29.648200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.662820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.668922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.695094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030cc5b-b3bd-4707-bf76-8bdbaeefcc18 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.735896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.736763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.737013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.737293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] PATH_INFO: `/v3/services/compute` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.737992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.740044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:29.752033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:29.753084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:29.753974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 497/987] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:29.759884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.765420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39977a62-88d0-4194-9082-3dd6a362d8a9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.810145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.811284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.811893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.811893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.823516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.824975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:29.843402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:29.844371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:29.845915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:29.845915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:29.845915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:29.845915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:29.845915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 491/988] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:29.851044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.884077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1c959-02af-4df8-8fe2-99bd036bbdff None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:29.917946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.921789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:29.922690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:29.922978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:29.923252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:29.923970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:29.924282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:29.925657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:29.936146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 498/989] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:29.936923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:29.941174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:29.966251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ffa59-5ced-45fb-933f-46491b4b4662 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.013522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.014225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.014519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.014796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.015511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:30.016249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.018003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.033760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 492/990] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:30.038899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:30.047272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b775d-2f17-41ff-a73b-d8c2fed54953 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.084166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.084758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.085048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.085330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.086074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:30.086386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.087754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.094381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 499/991] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:30.100847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:30.153944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53c84-c7bf-41e2-8f07-064d56885504 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.179855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:30.181041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:30.194845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.195722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.196015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.196298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.203657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.206233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:30.221375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:30.222453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:30.223633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:30.225795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc280d-699e-408f-80d4-bca127d5af99 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.227279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 493/992] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:30.228144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:30.273039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.273730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.274029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.274312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.275110 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] RBAC: Adding query filter params (interface=public, service_id=d2ac30ce65f54f8dba50890108a96cc1, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:30.275716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.276788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.282484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 500/993] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/endpoints?service_id=d2ac30ce65f54f8dba50890108a96cc1&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:30.288592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:30.353404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839d9c-0565-4f09-8422-f74b2fe662ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.392895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.393567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.393892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.394179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.395569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:30.395896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.397350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.398946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74246ad9-2f3f-4afd-9415-ddeed6445f5e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.410057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 494/994] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:30.417030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:30.438876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:30.445025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063e538-3ab3-4a75-b06f-2f05c6cd4da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.445702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063e538-3ab3-4a75-b06f-2f05c6cd4da4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.446009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063e538-3ab3-4a75-b06f-2f05c6cd4da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.446282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063e538-3ab3-4a75-b06f-2f05c6cd4da4 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.446990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063e538-3ab3-4a75-b06f-2f05c6cd4da4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.448385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063e538-3ab3-4a75-b06f-2f05c6cd4da4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.456195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 501/995] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:00:30.469324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:30.543080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93c643-b442-49c4-84c6-3ac843c456fd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:30.583554 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d99ef-2a13-4505-bb18-e8925d727b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:30.584239 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d99ef-2a13-4505-bb18-e8925d727b19 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.584528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d99ef-2a13-4505-bb18-e8925d727b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.584812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d99ef-2a13-4505-bb18-e8925d727b19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.585674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d99ef-2a13-4505-bb18-e8925d727b19 None admin] RBAC: Adding query filter params (role.id=66544664463c4fb9bf1c9d5bce6df901, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=bf00375b0f334f7bada8f21ea3766ac6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:30.586035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d99ef-2a13-4505-bb18-e8925d727b19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:30.587526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d99ef-2a13-4505-bb18-e8925d727b19 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:30.602120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 495/996] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/role_assignments?user.id=bf00375b0f334f7bada8f21ea3766ac6&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=66544664463c4fb9bf1c9d5bce6df901 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:30.807288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd76844-3653-457c-8ff5-1cb64eb04c25 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.807649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd76844-3653-457c-8ff5-1cb64eb04c25 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.807863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd76844-3653-457c-8ff5-1cb64eb04c25 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.809233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 502/997] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:30.814769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccc639-da79-4d67-a5e0-122dbd8bcead None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:30.815329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccc639-da79-4d67-a5e0-122dbd8bcead None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:30.815329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ccc639-da79-4d67-a5e0-122dbd8bcead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:30.832390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-45ccc639-da79-4d67-a5e0-122dbd8bcead None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:30.833592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-45ccc639-da79-4d67-a5e0-122dbd8bcead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:30.904888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 496/998] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:30.929469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd76844-3653-457c-8ff5-1cb64eb04c25 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:31.036303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd76844-3653-457c-8ff5-1cb64eb04c25 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:31.071289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:31.072018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:31.072304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:31.072584 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:31.073232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:31.074621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:31.087521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:31.088611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 16:00:31.089523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:31.091657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 503/999] 10.222.0.88 () {56 vars in 1126 bytes} [Thu Jun 16 16:00:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:31.094966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccc639-da79-4d67-a5e0-122dbd8bcead None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:31.140398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:31.204991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ccc639-da79-4d67-a5e0-122dbd8bcead None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:31.239906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:31.240573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:31.240854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:31.241129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:31.241860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:31.242169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:31.243533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:31.248642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 497/1000] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:31.254576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:31.316578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:31.361010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34381f40-e881-4cf9-bbbd-773ab51c2308 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:31.396182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418728c1-1ebd-472f-ac02-8f931d5b3521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:31.396856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418728c1-1ebd-472f-ac02-8f931d5b3521 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:31.397133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418728c1-1ebd-472f-ac02-8f931d5b3521 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:31.397410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418728c1-1ebd-472f-ac02-8f931d5b3521 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:31.398137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418728c1-1ebd-472f-ac02-8f931d5b3521 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:31.398489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418728c1-1ebd-472f-ac02-8f931d5b3521 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:31.399880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418728c1-1ebd-472f-ac02-8f931d5b3521 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:31.404935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 504/1001] 10.222.0.88 () {56 vars in 1122 bytes} [Thu Jun 16 16:00:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:31.411271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:31.514399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91f292-ce00-47bc-a0ba-dd377cd74995 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:31.516141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:31.550526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e97ed-f5bb-4577-855f-bee492e51b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:31.551151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e97ed-f5bb-4577-855f-bee492e51b3a None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:31.551435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e97ed-f5bb-4577-855f-bee492e51b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:31.551712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e97ed-f5bb-4577-855f-bee492e51b3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:31.552453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e97ed-f5bb-4577-855f-bee492e51b3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:31.553846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e97ed-f5bb-4577-855f-bee492e51b3a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:31.577775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 498/1002] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:32.627389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9dd01-61bb-470f-85fc-8447e5f7ca75 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.627949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9dd01-61bb-470f-85fc-8447e5f7ca75 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.627949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9dd01-61bb-470f-85fc-8447e5f7ca75 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.629460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 505/1003] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:32.635303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa3057-9091-4947-a8ce-2a0bc41d124a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.635746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa3057-9091-4947-a8ce-2a0bc41d124a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.635850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa3057-9091-4947-a8ce-2a0bc41d124a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.652995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d4fa3057-9091-4947-a8ce-2a0bc41d124a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:32.654241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d4fa3057-9091-4947-a8ce-2a0bc41d124a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:32.743539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 499/1004] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:32 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:32.766628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9dd01-61bb-470f-85fc-8447e5f7ca75 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:32.794674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e6fd1-6d28-4c02-bcc7-5c00b1deb6d5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.794958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e6fd1-6d28-4c02-bcc7-5c00b1deb6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.795239 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e6fd1-6d28-4c02-bcc7-5c00b1deb6d5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.796904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 500/1005] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:32.802190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4434-fccf-4c08-a857-4ebae768002f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.802464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4434-fccf-4c08-a857-4ebae768002f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.802739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae4434-fccf-4c08-a857-4ebae768002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.819848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-32ae4434-fccf-4c08-a857-4ebae768002f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:32.821012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-32ae4434-fccf-4c08-a857-4ebae768002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:32.876410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9dd01-61bb-470f-85fc-8447e5f7ca75 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:32.891972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 501/1006] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:32 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:32.902179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:32.914656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae4434-fccf-4c08-a857-4ebae768002f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:32.915629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:32.916427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:32.916708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:32.916985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:32.917629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:32.919119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:32.932612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:32.933850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:32.934698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 506/1007] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:32.940772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:32.941013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:33.029828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae4434-fccf-4c08-a857-4ebae768002f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.046205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bba57-c856-4ea7-81f9-16ca1b902d2e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.070597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.071979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.072574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.073224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.074289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.075977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.082408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.083563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.083844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.084124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.084855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:33.085223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.086621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:33.090875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:33.097158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:33.098100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:33.099594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 507/1008] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:33.099747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 502/1009] 10.222.0.88 () {56 vars in 1105 bytes} [Thu Jun 16 16:00:32 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:33.099747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:33.100011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:33.162460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:33.195487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:33.212900 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc043082-8e02-495d-b818-11b4e32c3a8a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.215550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28da35-9a52-46b1-b957-1b3a17293ce0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.247606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.248553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.248835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.249114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.249859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:33.250169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.251618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.254954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.255622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.255905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.256182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.256896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:33.257207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.258526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 508/1010] 10.222.0.88 () {56 vars in 1112 bytes} [Thu Jun 16 16:00:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:33.258726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.264127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:33.267323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 503/1011] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:33.273365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:33.375144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423cd2f-d97d-47c8-9f52-ffac296d88f5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.390039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3ef28-cb1c-4675-ba43-e735426ee030 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:33.392685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:33.412310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea2f92-a7d7-4e39-a5ec-7d0aea080fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.413025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea2f92-a7d7-4e39-a5ec-7d0aea080fc0 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.413257 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea2f92-a7d7-4e39-a5ec-7d0aea080fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.413531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea2f92-a7d7-4e39-a5ec-7d0aea080fc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.414480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea2f92-a7d7-4e39-a5ec-7d0aea080fc0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.415865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea2f92-a7d7-4e39-a5ec-7d0aea080fc0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.429028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905aba13-2334-47f4-a860-5e4122b3b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:33.429968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aba13-2334-47f4-a860-5e4122b3b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.430271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aba13-2334-47f4-a860-5e4122b3b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.430544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905aba13-2334-47f4-a860-5e4122b3b9a9 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.431470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aba13-2334-47f4-a860-5e4122b3b9a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:33.431858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aba13-2334-47f4-a860-5e4122b3b9a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:33.433413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905aba13-2334-47f4-a860-5e4122b3b9a9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:33.438491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 509/1012] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:33.438791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 504/1013] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:33 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:33.737411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb8dcb-329c-471a-9ea1-db6251ad9700 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.737939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb8dcb-329c-471a-9ea1-db6251ad9700 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.738040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb8dcb-329c-471a-9ea1-db6251ad9700 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.739873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 510/1014] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:33.745291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f81c2c-a4b6-4045-bbac-01128afbb088 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:33.745942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f81c2c-a4b6-4045-bbac-01128afbb088 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:33.745942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f81c2c-a4b6-4045-bbac-01128afbb088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:33.764545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c8f81c2c-a4b6-4045-bbac-01128afbb088 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:33.765831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8f81c2c-a4b6-4045-bbac-01128afbb088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:33.770509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:33.849772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 505/1015] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:33 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:33.872635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb8dcb-329c-471a-9ea1-db6251ad9700 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:33.982722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb8dcb-329c-471a-9ea1-db6251ad9700 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:34.019650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a65575-7970-47a1-b987-89f93b66be42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:34.020486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a65575-7970-47a1-b987-89f93b66be42 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:34.020770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a65575-7970-47a1-b987-89f93b66be42 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:34.021050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a65575-7970-47a1-b987-89f93b66be42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:34.022287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a65575-7970-47a1-b987-89f93b66be42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:34.023747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a65575-7970-47a1-b987-89f93b66be42 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:34.030491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 511/1016] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:34.032277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f81c2c-a4b6-4045-bbac-01128afbb088 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:34.143358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f81c2c-a4b6-4045-bbac-01128afbb088 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:34.179406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f9535-b20d-485a-823a-fde2327b8f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:34.180075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9535-b20d-485a-823a-fde2327b8f01 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:34.180359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9535-b20d-485a-823a-fde2327b8f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:34.180634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f9535-b20d-485a-823a-fde2327b8f01 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:34.181334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f9535-b20d-485a-823a-fde2327b8f01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:34.182814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f9535-b20d-485a-823a-fde2327b8f01 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:34.187544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:34.217762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 506/1017] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:34 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:35.603020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36687cc8-ffbe-4213-bd8d-99c263134d7d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.603550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36687cc8-ffbe-4213-bd8d-99c263134d7d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.603550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36687cc8-ffbe-4213-bd8d-99c263134d7d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.605089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 512/1018] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:35.610082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24cb36-be71-4ad8-8642-178b509c5e39 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.610478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24cb36-be71-4ad8-8642-178b509c5e39 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.610648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24cb36-be71-4ad8-8642-178b509c5e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.628741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5d24cb36-be71-4ad8-8642-178b509c5e39 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:35.630550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5d24cb36-be71-4ad8-8642-178b509c5e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:35.703285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 507/1019] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:35 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:35.723962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7b72-8df8-4b8a-aa0f-330ca7a1dbdf None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.724255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7b72-8df8-4b8a-aa0f-330ca7a1dbdf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.724541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7b72-8df8-4b8a-aa0f-330ca7a1dbdf None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.726250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24cb36-be71-4ad8-8642-178b509c5e39 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:35.726378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 513/1020] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:35.731487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df39d8b-4933-486d-9f88-4f02d032b51c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.731859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df39d8b-4933-486d-9f88-4f02d032b51c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.732215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df39d8b-4933-486d-9f88-4f02d032b51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.749608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2df39d8b-4933-486d-9f88-4f02d032b51c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:35.750941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2df39d8b-4933-486d-9f88-4f02d032b51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:35.787524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:35.829514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 514/1021] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:35 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:35.834221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24cb36-be71-4ad8-8642-178b509c5e39 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:35.852240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:35.853798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df39d8b-4933-486d-9f88-4f02d032b51c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:35.874158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:35.874840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:35.875178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:35.875461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:35.876187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:35.877620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:35.901751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 508/1022] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:00:35 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:00:35.965845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df39d8b-4933-486d-9f88-4f02d032b51c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.012547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.013372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.013653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.013966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.026102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:36.034401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:36.035901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:36.036874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:36.042929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 515/1023] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 16:00:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:36.043565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.160245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3268-0998-4d0b-a817-6272c4445fa8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.195807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.196488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.196773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.197052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.197840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:36.198138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.199523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.209381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 509/1024] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:36.214778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.243677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.322723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02465a-52ae-415a-9cd9-1f852a2f8f77 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.367826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e7417-bf11-4f41-9d87-37b8f821532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.368623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e7417-bf11-4f41-9d87-37b8f821532e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.368907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e7417-bf11-4f41-9d87-37b8f821532e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.369181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e7417-bf11-4f41-9d87-37b8f821532e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.380920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e7417-bf11-4f41-9d87-37b8f821532e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.382398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e7417-bf11-4f41-9d87-37b8f821532e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4a5e7417-bf11-4f41-9d87-37b8f821532e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:36.400137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:36.401034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:36.402469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:36.402469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:36.402469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:36.402469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:36.406001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 516/1025] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:36.407058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.460073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.494648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b6891-a401-4325-b917-2959c40f36da None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.494981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b6891-a401-4325-b917-2959c40f36da None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.495202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b6891-a401-4325-b917-2959c40f36da None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.496704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 517/1026] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:36.501499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6038-a1d7-464c-b072-42fe38f2856e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.501839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6038-a1d7-464c-b072-42fe38f2856e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.502098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd6038-a1d7-464c-b072-42fe38f2856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.518521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-46bd6038-a1d7-464c-b072-42fe38f2856e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:36.519685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-46bd6038-a1d7-464c-b072-42fe38f2856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:36.525695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860dfe3-ec90-439d-9ea4-fa1baa3a9c3f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.560459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.561129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.561407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.561681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.562449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:36.562767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.564119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.577542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 510/1027] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:36.583189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.599416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 518/1028] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:36.625134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd6038-a1d7-464c-b072-42fe38f2856e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.635152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.697344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.701395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2cb96-6d1a-47df-8872-c2b480f0371b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.733066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd6038-a1d7-464c-b072-42fe38f2856e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.743259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.744069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.744353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.744630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.753007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.754540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:36.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:36.771326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:36.772273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 511/1029] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:36.773827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.774641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.774923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.775199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.776416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.777155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.777876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.779731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 519/1030] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:36.786227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.867692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:36.885548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b417f-6aa9-4c75-95a8-5a14582bf934 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.892296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd77f00-3836-4bea-af79-1948cdacaa12 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:36.921341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.922128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.922380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.922659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.923481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:36.923790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.925228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.930376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:36.931238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:36.931670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:36.932111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:36.932812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:36.934375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:36.943926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 512/1031] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:36.949744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:36.961763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 520/1032] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 16:00:36 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:00:36.981201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:37.077276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ce6b-dd34-4dc6-8555-0bae5f72b7a9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:37.116756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:37.117431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:37.117712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:37.118016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:37.195437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:37.196898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:37.216334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:37.302466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 513/1033] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:37.308397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:37.413218 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0b536-06ac-4d6c-8304-1ab83e767596 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:37.442809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:37.449536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:37.450432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:37.451008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:37.451283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:37.464348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:37.465843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:37.483371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:37.484286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:37.485191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:37.485191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:37.485191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:37.485191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:37.489263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 521/1034] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:37.490457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:37.599675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525d8ba-9921-45f6-a848-dab65c078110 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:37.622730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:37.642085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:37.642756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:37.643039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:37.643316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:37.644033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:37.644341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:37.646353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:37.662893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 514/1035] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:37.664121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:37.774696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6e19f-0647-4991-b0de-bc7144e87690 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:37.811439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:37.812238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:37.812519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:37.812792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:37.819806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:37.821315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:37.836549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:37.837523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:37.838495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 522/1036] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:37.845411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:37.950989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6fd61-12dd-4b85-a93b-b01790ffeda9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:37.985890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f64da-21ac-4e48-814d-85846dadff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:37.986559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f64da-21ac-4e48-814d-85846dadff6d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:37.986844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f64da-21ac-4e48-814d-85846dadff6d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:37.987126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f64da-21ac-4e48-814d-85846dadff6d None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:37.987913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f64da-21ac-4e48-814d-85846dadff6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:37.988223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f64da-21ac-4e48-814d-85846dadff6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:37.990251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f64da-21ac-4e48-814d-85846dadff6d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:38.008424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 515/1037] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:38.009745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:38.065211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:38.126541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd22da-a10f-4922-909c-c7415b45d358 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:38.127535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778337b-9ccc-4f98-9a1e-af3d809bb2ea None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:38.127828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778337b-9ccc-4f98-9a1e-af3d809bb2ea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:38.128117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778337b-9ccc-4f98-9a1e-af3d809bb2ea None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:38.134135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 516/1038] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:38.135918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaa7e8-f0a5-4b0d-8874-575e3520f08a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:38.136207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaa7e8-f0a5-4b0d-8874-575e3520f08a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:38.136487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceaa7e8-f0a5-4b0d-8874-575e3520f08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:38.153981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1ceaa7e8-f0a5-4b0d-8874-575e3520f08a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:38.155180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ceaa7e8-f0a5-4b0d-8874-575e3520f08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:38.177366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:38.178077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:38.178364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:38.178644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:38.179495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] RBAC: Adding query filter params (role.id=66544664463c4fb9bf1c9d5bce6df901, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=ee056b132bd24851bd0e2afe9cb7170e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:38.179814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:38.181307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:38.188026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 523/1039] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:38 2022] GET /identity/v3/role_assignments?user.id=ee056b132bd24851bd0e2afe9cb7170e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=66544664463c4fb9bf1c9d5bce6df901 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:38.202854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:38.266009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 517/1040] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:38.289006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:38.413217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e352011-5e46-4fe6-a255-f0cd895ea9c5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:38.425646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:38.455224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:38.456289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:38.456732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:38.457152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] PATH_INFO: `/v3/services/image` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:38.458498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:38.460073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:38.472882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:38.474631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:38.475559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:38.481473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 524/1041] 10.222.0.88 () {56 vars in 1105 bytes} [Thu Jun 16 16:00:38 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:38.483055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceaa7e8-f0a5-4b0d-8874-575e3520f08a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:38.606868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceaa7e8-f0a5-4b0d-8874-575e3520f08a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:38.645821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:38.646835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:38.647307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:38.647781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:38.648679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:38.649169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:38.650963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:38.656640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 518/1042] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:38.662485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:38.778866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e5d6-adfc-4e26-8805-37902a740477 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:38.824690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:38.825615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:38.826068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:38.826510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:38.827362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] RBAC: Adding query filter params (type=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:38.827812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:38.829337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:38.836492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 525/1043] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:38.843089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:38.879666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:38.965879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95991316-d504-49c0-a8cb-7476098b4e25 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:39.009712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464da34a-66a0-4eb8-89d2-68c00f5b1c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:39.010925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da34a-66a0-4eb8-89d2-68c00f5b1c29 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:39.011498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da34a-66a0-4eb8-89d2-68c00f5b1c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:39.012107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464da34a-66a0-4eb8-89d2-68c00f5b1c29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:39.013173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464da34a-66a0-4eb8-89d2-68c00f5b1c29 None admin] RBAC: Adding query filter params (interface=public, service_id=7320b65c02804d0baae5b21f5adee1a4, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:39.013788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464da34a-66a0-4eb8-89d2-68c00f5b1c29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:39.015617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464da34a-66a0-4eb8-89d2-68c00f5b1c29 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:39.022885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 519/1044] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:00:38 2022] GET /identity/v3/endpoints?service_id=7320b65c02804d0baae5b21f5adee1a4&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:39.029852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:39.075356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:39.157088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe59c2-185e-4bae-ad93-698267014c57 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:39.210175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352681c3-ec30-44c7-97b0-03e98c2221c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:39.211008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352681c3-ec30-44c7-97b0-03e98c2221c0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:39.211372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352681c3-ec30-44c7-97b0-03e98c2221c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:39.211683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352681c3-ec30-44c7-97b0-03e98c2221c0 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:39.212531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352681c3-ec30-44c7-97b0-03e98c2221c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:39.214388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352681c3-ec30-44c7-97b0-03e98c2221c0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:39.230027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 526/1045] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:00:39 2022] GET /identity/v3/services => generated 1957 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:00:39.252562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322b4a8-282c-436d-81ba-2a007cc4423e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:39.252865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322b4a8-282c-436d-81ba-2a007cc4423e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:39.253132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322b4a8-282c-436d-81ba-2a007cc4423e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:39.254673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 520/1046] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:39.261333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a659d-f703-451c-a678-8ca25babc8df None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:39.261621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a659d-f703-451c-a678-8ca25babc8df None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:39.261946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a659d-f703-451c-a678-8ca25babc8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:39.284573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2c8a659d-f703-451c-a678-8ca25babc8df None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:39.286073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2c8a659d-f703-451c-a678-8ca25babc8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:39.363055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 527/1047] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:39.387393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322b4a8-282c-436d-81ba-2a007cc4423e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:39.506682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:39.514738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322b4a8-282c-436d-81ba-2a007cc4423e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:39.557437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:39.558193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:39.558480 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:39.558760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:39.572160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:39.580279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:39.581268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:39.582153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 521/1048] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:39.588667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a659d-f703-451c-a678-8ca25babc8df None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:39.622132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:39.705758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a659d-f703-451c-a678-8ca25babc8df None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:39.746703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:39.747391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:39.747677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:39.748024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:39.748708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:39.749015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:39.750718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:39.763893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 528/1049] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:39.770629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:39.889045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3627e-0ddb-44fa-8606-42213f466ad6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:39.946502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:39.947423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:39.947713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:39.947999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] PATH_INFO: `/v3/users/166a7695a987429d98b1e0abe8bee2ec` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:39.967899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] RBAC: Authorizing `identity:get_user(user_id=166a7695a987429d98b1e0abe8bee2ec)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:39.968717 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:39.971232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 522/1050] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:39 2022] GET /identity/v3/users/166a7695a987429d98b1e0abe8bee2ec => generated 285 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:39.977189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:40.094686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:40.115613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5c500-1e5f-4758-ade7-e775c0f94a69 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:40.172422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee48a5-7a59-4f5a-a697-4960697acfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:40.173087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee48a5-7a59-4f5a-a697-4960697acfec None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:40.173401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee48a5-7a59-4f5a-a697-4960697acfec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:40.173812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee48a5-7a59-4f5a-a697-4960697acfec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:40.263691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee48a5-7a59-4f5a-a697-4960697acfec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:40.265342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee48a5-7a59-4f5a-a697-4960697acfec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:40.366171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:40.381987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 529/1051] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:39 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:00:40.385669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:40.415055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:40.462002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738bb73-4765-4e2a-8e6c-9c4690ba8e9c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:40.462186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738bb73-4765-4e2a-8e6c-9c4690ba8e9c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:40.462476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738bb73-4765-4e2a-8e6c-9c4690ba8e9c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:40.463983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 530/1052] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:40.469294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc286d-582a-4dfd-b1fa-3e730c0cc013 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:40.469630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc286d-582a-4dfd-b1fa-3e730c0cc013 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:40.469899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc286d-582a-4dfd-b1fa-3e730c0cc013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:40.486715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-17fc286d-582a-4dfd-b1fa-3e730c0cc013 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:40.487906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-17fc286d-582a-4dfd-b1fa-3e730c0cc013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:40.512937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb634b6-1a5a-4f3f-8490-a23bb905340c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:40.552475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:40.553346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:40.553660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:40.553962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:40.555135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] RBAC: Authorizing `identity:get_project(project_id=713479262ec84f0f86df3bbb9c1dc822)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:40.556601 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:40.563306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 523/1053] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:40 2022] GET /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:40.564203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:40.567872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 531/1054] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:40.591106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc286d-582a-4dfd-b1fa-3e730c0cc013 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:40.655386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:40.676881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5965aff-7b7f-4e29-be22-dd29056fb6d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:40.706022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc286d-582a-4dfd-b1fa-3e730c0cc013 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:40.713824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:40.714417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:40.714707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:40.714991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:40.715866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=713479262ec84f0f86df3bbb9c1dc822, user.id=166a7695a987429d98b1e0abe8bee2ec) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:40.716188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:40.717941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:40.733750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 524/1055] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:40 2022] GET /identity/v3/role_assignments?user.id=166a7695a987429d98b1e0abe8bee2ec&scope.project.id=713479262ec84f0f86df3bbb9c1dc822&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:40.734010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:40.747866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:40.748706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:40.748996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:40.749278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:40.762536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:40.770935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:40.772144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:40.773041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 532/1056] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 16:00:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:40.779227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:40.899305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20ed5e-37a8-437e-a9ff-59895df892da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:40.936970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:40.938240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:40.938743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:40.939240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:40.940184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:40.940706 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:40.942840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:40.956390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 525/1057] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 16:00:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:40.958213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.070067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ce603-0b7c-43dd-80c4-eab49f9607d6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:41.104473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:41.105262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.105543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.105845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.117953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:41.119411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:41.138345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:41.139352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:41.140793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:41.140793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:41.140793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:41.140793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:41.142530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 533/1058] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:41.149371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.164087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:41.262318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bc729-299c-4675-a560-93eec5910d63 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:41.303025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:41.303693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.303981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.304261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.304984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:41.305296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:41.306956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:41.320690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 526/1059] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:41.326016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.364822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:41.444348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47539555-45b1-42ea-a45b-b924a96c35fb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:41.486457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4858ab-b6db-48e6-9604-abf35002664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:41.487254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4858ab-b6db-48e6-9604-abf35002664e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.487589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4858ab-b6db-48e6-9604-abf35002664e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.487842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4858ab-b6db-48e6-9604-abf35002664e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.496143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4858ab-b6db-48e6-9604-abf35002664e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:41.497591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4858ab-b6db-48e6-9604-abf35002664e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-dd4858ab-b6db-48e6-9604-abf35002664e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:41.512768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:41.515892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:41.516846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:41.518962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 534/1060] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 16:00:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:41.520537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.529450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2cae-e3e8-4f91-a3cc-c993041cb3ae None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.529778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2cae-e3e8-4f91-a3cc-c993041cb3ae None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.530073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2cae-e3e8-4f91-a3cc-c993041cb3ae None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.531703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 535/1061] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:41.536923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a1e33-a63c-47dd-9739-c36ef6f54e6d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.537253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a1e33-a63c-47dd-9739-c36ef6f54e6d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.537572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a1e33-a63c-47dd-9739-c36ef6f54e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.555326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-828a1e33-a63c-47dd-9739-c36ef6f54e6d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:41.556509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-828a1e33-a63c-47dd-9739-c36ef6f54e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:41.593906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:41.630555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5bd6e-1733-4579-8c76-5b8fc8ff80b1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:41.637754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 536/1062] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:41.660086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a1e33-a63c-47dd-9739-c36ef6f54e6d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.666202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:41.666883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.667164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.667440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.668269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:41.668580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:41.670056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:41.693623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 527/1063] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 16:00:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:41.693863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:41.694730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.778009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a1e33-a63c-47dd-9739-c36ef6f54e6d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:41.810741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365271c-60cf-4b4b-9473-0ed2a5bd1690 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:41.825134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:41.826266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.826659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.827102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] PATH_INFO: `/v3/services/image` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.828292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:41.831231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:41.844200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:41.850380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:41.851536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 16:00:41.852392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:41.853150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 537/1064] 10.222.0.88 () {56 vars in 1105 bytes} [Thu Jun 16 16:00:41 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:41.858645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.861307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:41.862162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:41.862445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:41.862716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:41.882138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:41.883768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:41.903407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:41.904830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:41.905775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:41.905775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:41.905775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:41.905775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:41.905775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 528/1065] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:41.912048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:41.975314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:41.997021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f6582-c651-4593-8122-8497d82c9b1b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:42.034349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:42.035186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.036024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.036378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.037203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] RBAC: Adding query filter params (name=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:42.037511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:42.039090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:42.039721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c986b-0244-469a-8104-f15e8244eda8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:42.044709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 538/1066] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:42.050503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:42.081825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:42.083406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.083845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.084401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.085278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:42.085753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:42.087495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:42.100890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 529/1067] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:42.105970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:42.160632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:42.183723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7f8a4-76b4-4b23-83c2-fa2074f48399 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:42.222178 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:42.222910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.223209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.223517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.224210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:42.224524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:42.225923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:42.227547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a7f0e-cc1b-495c-98b0-f6fc02f0b6cb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:42.231347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 539/1068] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:42.237026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:42.257198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:42.265326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:42.266369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.266865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.266985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.275496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:42.276981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:42.292463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:42.293536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:42.295106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 530/1069] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:42.299498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:42.357480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f64ad-0077-4c00-a1ac-9755ea2dccf8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:42.392412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:42.393082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.393370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.393654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.394426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:42.396364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:42.419239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 540/1070] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:00:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:00:42.425855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50eb7b-7696-4535-84f0-7325e74cda24 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:42.430336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:42.484177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd685e7-a4b0-407d-897e-96f099cd9899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:42.484800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd685e7-a4b0-407d-897e-96f099cd9899 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.485060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd685e7-a4b0-407d-897e-96f099cd9899 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.485341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd685e7-a4b0-407d-897e-96f099cd9899 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.486168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd685e7-a4b0-407d-897e-96f099cd9899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:42.486485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd685e7-a4b0-407d-897e-96f099cd9899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:42.487960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd685e7-a4b0-407d-897e-96f099cd9899 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:42.501578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 531/1071] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:42.507715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:42.588365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:42.629847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219f25c-acfc-458a-a2ae-dcd3c7285041 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:42.667599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000eef4-6f24-4d75-9106-66a17d17cb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:42.668606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000eef4-6f24-4d75-9106-66a17d17cb03 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.668895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000eef4-6f24-4d75-9106-66a17d17cb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.669171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000eef4-6f24-4d75-9106-66a17d17cb03 None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/ee056b132bd24851bd0e2afe9cb7170e/roles/66544664463c4fb9bf1c9d5bce6df901` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.671373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000eef4-6f24-4d75-9106-66a17d17cb03 None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=ee056b132bd24851bd0e2afe9cb7170e, role_id=66544664463c4fb9bf1c9d5bce6df901)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:42.674650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000eef4-6f24-4d75-9106-66a17d17cb03 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:42.687555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 541/1072] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:42 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/ee056b132bd24851bd0e2afe9cb7170e/roles/66544664463c4fb9bf1c9d5bce6df901 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:42.959101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa34f4f-d8a6-4010-98be-19491beaa03a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.959609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa34f4f-d8a6-4010-98be-19491beaa03a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.959609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa34f4f-d8a6-4010-98be-19491beaa03a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.960995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 532/1073] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:42.966387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5f73a-cae2-4f96-af1f-3979c59702b3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:42.966682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5f73a-cae2-4f96-af1f-3979c59702b3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:42.966962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5f73a-cae2-4f96-af1f-3979c59702b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:42.984738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ecb5f73a-cae2-4f96-af1f-3979c59702b3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:42.986258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ecb5f73a-cae2-4f96-af1f-3979c59702b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:43.063971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 542/1074] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:43.087296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa34f4f-d8a6-4010-98be-19491beaa03a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:43.203655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa34f4f-d8a6-4010-98be-19491beaa03a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:43.239127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:43.239857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:43.240143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:43.240423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:43.251895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:43.260027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:43.261178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:43.267226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 533/1075] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:43.269571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5f73a-cae2-4f96-af1f-3979c59702b3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:43.334581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:43.377001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5f73a-cae2-4f96-af1f-3979c59702b3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:43.413282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:43.413989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:43.414274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:43.414551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:43.415295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:43.415602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:43.416964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:43.431124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 543/1076] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:43.432501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:43.450006 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:43.540824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bdca5-7c1b-4c73-9b7d-0e1f5845c7d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:43.577913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:43.578806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:43.579093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:43.579377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] PATH_INFO: `/v3/users/166a7695a987429d98b1e0abe8bee2ec` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:43.580588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] RBAC: Authorizing `identity:get_user(user_id=166a7695a987429d98b1e0abe8bee2ec)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:43.582071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:43.584520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 534/1077] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:43 2022] GET /identity/v3/users/166a7695a987429d98b1e0abe8bee2ec => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:43.589968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:43.696003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541984e6-9e22-404b-aec9-976339fff331 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:43.720151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:43.732095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78f93a-0176-418a-bab2-e0749ed4bea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:43.732958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78f93a-0176-418a-bab2-e0749ed4bea5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:43.733242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78f93a-0176-418a-bab2-e0749ed4bea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:43.733521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78f93a-0176-418a-bab2-e0749ed4bea5 None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:43.734911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78f93a-0176-418a-bab2-e0749ed4bea5 None admin] RBAC: Authorizing `identity:get_project(project_id=713479262ec84f0f86df3bbb9c1dc822)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:43.736355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78f93a-0176-418a-bab2-e0749ed4bea5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:43.743473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 544/1078] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:43 2022] GET /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:43.744980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:43.833085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:43.856130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c2ba3-47ce-41e1-bb55-c283677a6805 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:43.892713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4bf54-0e92-470f-8a07-e066bd46efbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:43.893604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4bf54-0e92-470f-8a07-e066bd46efbb None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:43.893927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4bf54-0e92-470f-8a07-e066bd46efbb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:43.894261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4bf54-0e92-470f-8a07-e066bd46efbb None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822/users/166a7695a987429d98b1e0abe8bee2ec/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:43.896768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4bf54-0e92-470f-8a07-e066bd46efbb None admin] RBAC: Authorizing `identity:create_grant(project_id=713479262ec84f0f86df3bbb9c1dc822, user_id=166a7695a987429d98b1e0abe8bee2ec, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:43.898274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c4bf54-0e92-470f-8a07-e066bd46efbb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:43.913280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 535/1079] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:43 2022] PUT /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822/users/166a7695a987429d98b1e0abe8bee2ec/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:44.618671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5861d-f493-4045-9fd7-737f4fc70ea5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:44.619220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5861d-f493-4045-9fd7-737f4fc70ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:44.619220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5861d-f493-4045-9fd7-737f4fc70ea5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:44.620572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 545/1080] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:44.625938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef491c1-c442-400f-81b1-abdd56a9ade9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:44.626372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef491c1-c442-400f-81b1-abdd56a9ade9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:44.626580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef491c1-c442-400f-81b1-abdd56a9ade9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:44.643182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bef491c1-c442-400f-81b1-abdd56a9ade9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:44.644362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bef491c1-c442-400f-81b1-abdd56a9ade9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:44.722695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 536/1081] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:44.745648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5861d-f493-4045-9fd7-737f4fc70ea5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:44.854906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5861d-f493-4045-9fd7-737f4fc70ea5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:44.891473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:44.892063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:44.892348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:44.892624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:44.953528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32c1d4-24b4-4eda-be86-9e1442f7e040 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:44.953870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32c1d4-24b4-4eda-be86-9e1442f7e040 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:44.954119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32c1d4-24b4-4eda-be86-9e1442f7e040 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:44.955032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:44.955529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 537/1082] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:44.960851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bc7a1-432f-4768-83c2-16deb5ba31ef None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:44.961171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bc7a1-432f-4768-83c2-16deb5ba31ef None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:44.961420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bc7a1-432f-4768-83c2-16deb5ba31ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:44.966783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:44.969348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:44.979265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c1bc7a1-432f-4768-83c2-16deb5ba31ef None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:44.980456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c1bc7a1-432f-4768-83c2-16deb5ba31ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:45.061770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 538/1083] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:45.087427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bc7a1-432f-4768-83c2-16deb5ba31ef None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:45.089575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 546/1084] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:45.089875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:45.095607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:45.199846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:45.209747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314b610-a0f8-43fc-94eb-5bc66f735f3e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:45.210521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bc7a1-432f-4768-83c2-16deb5ba31ef None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:45.246976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:45.247777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:45.248060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:45.248331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:45.248880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:45.249806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:45.250095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:45.250390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:45.259756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:45.261386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:45.262892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:45.267864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:45.268832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:45.269918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 539/1085] 10.222.0.88 () {56 vars in 1120 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:45.274874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:45.281256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:45.282215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:45.283793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:45.283793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:45.283793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:45.283793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:45.283793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 547/1086] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:45.293828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:45.340621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:45.383407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab367e5e-b07e-4f9b-b416-abba3269d0b2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:45.412810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77ef0a-48e7-4b80-a4c1-c144ca786ada None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:45.421468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:45.422397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:45.422675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:45.422955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:45.423693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:45.424167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:45.425375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:45.434949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 540/1087] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:45.440955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:45.451506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:45.452101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:45.452387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:45.452661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:45.453384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:45.453747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:45.455114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:45.468229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 548/1088] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:45.563151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b496a9-5816-4de5-978e-04b71b4c4ac3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:45.589598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:45.603670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:45.604486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:45.604770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:45.605044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:45.617243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:45.618738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:45.637555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:45.639236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:45.640432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:45.640432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:45.640432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:45.640432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:45.640432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 541/1089] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:45.645290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:45.649662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:45.763031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ace87e-bc16-4d66-9c74-e3dc6221f2ef None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:45.800428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:45.801660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:45.801988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:45.802260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:45.802982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:45.803281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:45.804650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:45.823726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 549/1090] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:45.825103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:45.932127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cbbcc6-ce3e-4272-baaa-02cab1850564 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:45.966341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013424cf-6d79-47a2-ba61-fa0ca47869dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:45.967132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013424cf-6d79-47a2-ba61-fa0ca47869dd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:45.967668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013424cf-6d79-47a2-ba61-fa0ca47869dd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:45.967952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013424cf-6d79-47a2-ba61-fa0ca47869dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:45.976042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013424cf-6d79-47a2-ba61-fa0ca47869dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:45.977484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013424cf-6d79-47a2-ba61-fa0ca47869dd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-013424cf-6d79-47a2-ba61-fa0ca47869dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:00:45.992079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:45.993120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:45.994159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 542/1091] 10.222.0.88 () {56 vars in 1093 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:00:45.998986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.065326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.093643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d9c4a-a662-4997-99e2-becedfcc0ed0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.093960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d9c4a-a662-4997-99e2-becedfcc0ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.094246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d9c4a-a662-4997-99e2-becedfcc0ed0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.095787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 543/1092] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:00:46.102555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109447b-157d-4a49-ad1b-532baa246199 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.102857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109447b-157d-4a49-ad1b-532baa246199 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.103147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109447b-157d-4a49-ad1b-532baa246199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.105961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509ffce-6619-4146-ba4a-7feb22fe3e4b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.120766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8109447b-157d-4a49-ad1b-532baa246199 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:46.122209 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8109447b-157d-4a49-ad1b-532baa246199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:46.141619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.142325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.142608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.142887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.143721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:46.144032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.145995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.154255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.164067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 550/1093] 10.222.0.88 () {56 vars in 1133 bytes} [Thu Jun 16 16:00:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:46.169970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.195141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 544/1094] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:00:46.219744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109447b-157d-4a49-ad1b-532baa246199 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.272828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0bbd-a785-4f23-abf3-36263d5bcec1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.297547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.310935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.311593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.311878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.312158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.326284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109447b-157d-4a49-ad1b-532baa246199 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.361105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.362079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.362369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.362658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.372825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:00:46.381165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:46.382389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:00:46.383471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:00:46.384222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.384308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 545/1095] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:46.384387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.388492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.417631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.483100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 551/1096] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:00:46.489079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.493492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec2f56-c4db-4da0-88fd-e40e42d1629b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.514536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.532636 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.533757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.534099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.534384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.535524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:46.535836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.537343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.550558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 546/1097] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:46.553203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.597099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe9f75-4c2d-43b0-b153-38bbbec23942 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.631585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.632712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.632946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.633532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.642415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.644251 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.646238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.656471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb5ccb-f8d3-4188-b038-dd311c62e7f6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:00:46.664695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:46.665663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:46.666744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:00:46.666744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:00:46.666744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:00:46.666744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:46.666744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 552/1098] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:46.671571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.694642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.695440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.695723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.695993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] PATH_INFO: `/v3/users/166a7695a987429d98b1e0abe8bee2ec` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.697206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] RBAC: Authorizing `identity:get_user(user_id=166a7695a987429d98b1e0abe8bee2ec)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.698690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.700604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 547/1099] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/users/166a7695a987429d98b1e0abe8bee2ec => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:46.705996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.751298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.777687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2fa54-c460-4084-bc09-ba4175e4047d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.811974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.812629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.812909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.813185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.813921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:46.814223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.815704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.817381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866950ee-40f5-4e2e-b65e-130a97e130a1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.827736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 553/1100] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:46.834329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:46.853225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.853789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.854086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.854365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.879947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.924855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.926871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.935427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa89ca-c68c-4e1c-9030-fda6c9043a74 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:46.969983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:46.970862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:46.971154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:46.971437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:46.973400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:46.978906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:46.980571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:00:46.995126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:00:46.996545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:00:46.997787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 554/1101] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:00:47.002309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:47.029140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 548/1102] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:00:47.034949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:47.091448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:47.104371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03ee0e-6b2b-420c-a924-89c807d60e28 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:47.136526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f898a51-e348-4121-80c8-0ec55bb0af08 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:47.143913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:47.144580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:47.144857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:47.145130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:47.145950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:47.146259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:47.147681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:47.159357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 555/1103] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:47.165818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:47.171690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:47.172550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:47.172832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:47.173110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:47.174367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] RBAC: Authorizing `identity:get_project(project_id=713479262ec84f0f86df3bbb9c1dc822)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:47.175837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:47.178178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 549/1104] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:47 2022] GET /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:47.184021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:47.219980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:47.266850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223ea40-8802-4108-b9c0-711a6a718e5f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:47.284927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589cf29-752b-4d6c-a738-ed43c836d952 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:47.301512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342e56c-d736-41f2-82b4-e38759214547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:47.302206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e56c-d736-41f2-82b4-e38759214547 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:47.302492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e56c-d736-41f2-82b4-e38759214547 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:47.302768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342e56c-d736-41f2-82b4-e38759214547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:47.303587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342e56c-d736-41f2-82b4-e38759214547 None admin] RBAC: Adding query filter params (role.id=66544664463c4fb9bf1c9d5bce6df901, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=ee056b132bd24851bd0e2afe9cb7170e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:47.303896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342e56c-d736-41f2-82b4-e38759214547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:47.305311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342e56c-d736-41f2-82b4-e38759214547 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:47.309692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:47.311081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 556/1105] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:47 2022] GET /identity/v3/role_assignments?user.id=ee056b132bd24851bd0e2afe9cb7170e&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=66544664463c4fb9bf1c9d5bce6df901 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:00:47.321933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bd899-81ea-41b2-b0f2-da341856c9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:47.322618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bd899-81ea-41b2-b0f2-da341856c9df None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:47.322902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bd899-81ea-41b2-b0f2-da341856c9df None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:47.323181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bd899-81ea-41b2-b0f2-da341856c9df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:47.324027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bd899-81ea-41b2-b0f2-da341856c9df None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=713479262ec84f0f86df3bbb9c1dc822, user.id=166a7695a987429d98b1e0abe8bee2ec) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:47.324346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bd899-81ea-41b2-b0f2-da341856c9df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:47.325966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bd899-81ea-41b2-b0f2-da341856c9df None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:47.333486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 550/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:47 2022] GET /identity/v3/role_assignments?user.id=166a7695a987429d98b1e0abe8bee2ec&scope.project.id=713479262ec84f0f86df3bbb9c1dc822&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:49.521033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329df50-5ff5-42f7-abbb-b1967a6b7b79 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:49.521583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329df50-5ff5-42f7-abbb-b1967a6b7b79 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:49.521583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329df50-5ff5-42f7-abbb-b1967a6b7b79 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:49.522969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 557/1107] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:49.528332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da087cdd-b1be-4f45-875e-25d95d22925c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:49.528731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da087cdd-b1be-4f45-875e-25d95d22925c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:49.528902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da087cdd-b1be-4f45-875e-25d95d22925c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:49.545855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da087cdd-b1be-4f45-875e-25d95d22925c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:49.547057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da087cdd-b1be-4f45-875e-25d95d22925c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:49.613631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 551/1108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:49.639229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329df50-5ff5-42f7-abbb-b1967a6b7b79 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:49.741039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329df50-5ff5-42f7-abbb-b1967a6b7b79 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:49.758057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:49.774298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a79a4-3d56-473d-8176-6243569eb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:49.775014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a79a4-3d56-473d-8176-6243569eb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:49.775294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a79a4-3d56-473d-8176-6243569eb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:49.775570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a79a4-3d56-473d-8176-6243569eb4f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:49.776716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a79a4-3d56-473d-8176-6243569eb4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:49.778171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a79a4-3d56-473d-8176-6243569eb4f0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:49.779771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 558/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:00:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:49.785514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da087cdd-b1be-4f45-875e-25d95d22925c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:49.887714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da087cdd-b1be-4f45-875e-25d95d22925c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:49.906357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:49.923221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ae131-7e65-4472-a952-e0256b596bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:49.923886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae131-7e65-4472-a952-e0256b596bfd None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:49.924168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae131-7e65-4472-a952-e0256b596bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:49.924441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ae131-7e65-4472-a952-e0256b596bfd None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:49.925157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae131-7e65-4472-a952-e0256b596bfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:49.927130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ae131-7e65-4472-a952-e0256b596bfd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:49.947281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 552/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 16:00:49 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:00:52.127228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4a272-e9e1-44dd-b98b-903bb1fdf60d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:52.127798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4a272-e9e1-44dd-b98b-903bb1fdf60d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:52.127798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4a272-e9e1-44dd-b98b-903bb1fdf60d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:52.129201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 559/1111] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:52.134838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d7c12-2fc1-47ad-83ae-1ccce7989426 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:52.135312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d7c12-2fc1-47ad-83ae-1ccce7989426 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:52.135410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d7c12-2fc1-47ad-83ae-1ccce7989426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:52.152906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-557d7c12-2fc1-47ad-83ae-1ccce7989426 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:52.154153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-557d7c12-2fc1-47ad-83ae-1ccce7989426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:52.221200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 553/1112] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:52.245183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4a272-e9e1-44dd-b98b-903bb1fdf60d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:52.351673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4a272-e9e1-44dd-b98b-903bb1fdf60d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:52.385078 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:52.385865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:52.386146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:52.386421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] PATH_INFO: `/v3/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:52.387862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] RBAC: Authorizing `identity:get_role(role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:52.389216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:52.390697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 560/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:52 2022] GET /identity/v3/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:52.397078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d7c12-2fc1-47ad-83ae-1ccce7989426 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:52.495972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d7c12-2fc1-47ad-83ae-1ccce7989426 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:52.498300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:52.531352 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:52.532227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:52.532516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:52.532800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] PATH_INFO: `/v3/users/7f709911b8ae455fb6df71f32e237a58` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:52.546720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] RBAC: Authorizing `identity:get_user(user_id=7f709911b8ae455fb6df71f32e237a58)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:52.548177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:52.549893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 554/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:52 2022] GET /identity/v3/users/7f709911b8ae455fb6df71f32e237a58 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:52.555742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:52.611782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:52.655803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111424e-659e-42cd-bfe3-8d37a3d2a36c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:52.689083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:52.689754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:52.690032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:52.690306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:52.759891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:52.761317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:52.812835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:52.858722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 561/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:52.864699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:52.967844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1783e-afb3-43bf-81bd-61023c447bea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:53.001092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f1aba-7ef3-4267-81aa-4169ba1b247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:53.001971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f1aba-7ef3-4267-81aa-4169ba1b247d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:53.002260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f1aba-7ef3-4267-81aa-4169ba1b247d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:53.002541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f1aba-7ef3-4267-81aa-4169ba1b247d None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:53.003724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f1aba-7ef3-4267-81aa-4169ba1b247d None admin] RBAC: Authorizing `identity:get_project(project_id=713479262ec84f0f86df3bbb9c1dc822)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:53.005728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f1aba-7ef3-4267-81aa-4169ba1b247d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:53.007452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 555/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:52 2022] GET /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:53.013534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:53.111881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1708a-ffbb-49bc-a001-335b7b533756 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:53.147198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bead0b-d988-4d0d-9fcf-e5e97d57a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:53.147851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bead0b-d988-4d0d-9fcf-e5e97d57a33e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:53.148131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bead0b-d988-4d0d-9fcf-e5e97d57a33e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:53.148403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bead0b-d988-4d0d-9fcf-e5e97d57a33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:53.149363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bead0b-d988-4d0d-9fcf-e5e97d57a33e None admin] RBAC: Adding query filter params (role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=713479262ec84f0f86df3bbb9c1dc822, user.id=7f709911b8ae455fb6df71f32e237a58) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:53.149677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bead0b-d988-4d0d-9fcf-e5e97d57a33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:53.151117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bead0b-d988-4d0d-9fcf-e5e97d57a33e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:53.156650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 562/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:53 2022] GET /identity/v3/role_assignments?user.id=7f709911b8ae455fb6df71f32e237a58&scope.project.id=713479262ec84f0f86df3bbb9c1dc822&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:55.257977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a57b-88cd-44ff-9663-f3eabdefef98 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:55.258524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a57b-88cd-44ff-9663-f3eabdefef98 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:55.258524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2a57b-88cd-44ff-9663-f3eabdefef98 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:55.259903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 556/1118] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:55.265300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300395-d450-4f6b-a9c1-2b33664773d1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:55.265734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300395-d450-4f6b-a9c1-2b33664773d1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:55.265888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91300395-d450-4f6b-a9c1-2b33664773d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:55.282150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-91300395-d450-4f6b-a9c1-2b33664773d1 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:55.283322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-91300395-d450-4f6b-a9c1-2b33664773d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:55.296546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:55.350130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 563/1119] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:55.373229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a57b-88cd-44ff-9663-f3eabdefef98 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:55.403708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:55.475813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2a57b-88cd-44ff-9663-f3eabdefef98 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:55.508685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:55.509531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:55.509845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:55.510125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] PATH_INFO: `/v3/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:55.511582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] RBAC: Authorizing `identity:get_role(role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:55.512951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:55.514358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 557/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:55 2022] GET /identity/v3/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:55.520335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91300395-d450-4f6b-a9c1-2b33664773d1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:55.616514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91300395-d450-4f6b-a9c1-2b33664773d1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:55.649425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:55.650305 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:55.650592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:55.650862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] PATH_INFO: `/v3/users/7f709911b8ae455fb6df71f32e237a58` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:55.652015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] RBAC: Authorizing `identity:get_user(user_id=7f709911b8ae455fb6df71f32e237a58)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:55.653431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:55.655113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 564/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:55 2022] GET /identity/v3/users/7f709911b8ae455fb6df71f32e237a58 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:55.661633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:55.756766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:55.760891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d4978-aef2-44fe-b365-ccd31c518a2a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:55.794686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37918ff8-c9e7-4645-83c1-85527b9a1db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:55.795537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37918ff8-c9e7-4645-83c1-85527b9a1db2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:55.795823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37918ff8-c9e7-4645-83c1-85527b9a1db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:55.796191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37918ff8-c9e7-4645-83c1-85527b9a1db2 None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:55.797353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37918ff8-c9e7-4645-83c1-85527b9a1db2 None admin] RBAC: Authorizing `identity:get_project(project_id=713479262ec84f0f86df3bbb9c1dc822)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:55.798819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37918ff8-c9e7-4645-83c1-85527b9a1db2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:55.800520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 558/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:55 2022] GET /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:55.807163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:55.825303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:55.917676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cd1db-1276-49ca-bcba-9f984fc190a5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:55.954892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87843a97-b068-4743-b08a-63709384af02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:55.955781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87843a97-b068-4743-b08a-63709384af02 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:55.956058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87843a97-b068-4743-b08a-63709384af02 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:55.956330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87843a97-b068-4743-b08a-63709384af02 None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822/users/7f709911b8ae455fb6df71f32e237a58/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:55.958603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87843a97-b068-4743-b08a-63709384af02 None admin] RBAC: Authorizing `identity:create_grant(project_id=713479262ec84f0f86df3bbb9c1dc822, user_id=7f709911b8ae455fb6df71f32e237a58, role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:55.960037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87843a97-b068-4743-b08a-63709384af02 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:55.974564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 565/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:00:55 2022] PUT /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822/users/7f709911b8ae455fb6df71f32e237a58/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:00:58.116769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c0043-410d-4ece-999e-527c8e9e150e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:58.117300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c0043-410d-4ece-999e-527c8e9e150e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:58.117414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c0043-410d-4ece-999e-527c8e9e150e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:58.118763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 559/1124] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:00:58.123766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862d306-6e69-4817-a75f-ca996145110e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:58.124341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862d306-6e69-4817-a75f-ca996145110e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:58.124341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862d306-6e69-4817-a75f-ca996145110e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:58.141780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6862d306-6e69-4817-a75f-ca996145110e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:00:58.142974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6862d306-6e69-4817-a75f-ca996145110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:00:58.207527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 566/1125] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:00:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:00:58.231486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c0043-410d-4ece-999e-527c8e9e150e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:58.330539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c0043-410d-4ece-999e-527c8e9e150e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:58.363504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:58.364345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:58.364633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:58.364909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] PATH_INFO: `/v3/roles/b59592ce5fcf4e3981fd342b5352ee2b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:58.366385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] RBAC: Authorizing `identity:get_role(role_id=b59592ce5fcf4e3981fd342b5352ee2b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:58.367811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:58.369131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 560/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:58 2022] GET /identity/v3/roles/b59592ce5fcf4e3981fd342b5352ee2b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:58.374918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862d306-6e69-4817-a75f-ca996145110e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:58.383500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:58.474570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862d306-6e69-4817-a75f-ca996145110e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:58.508462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:58.509321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:58.509599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:58.509899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] PATH_INFO: `/v3/users/7f709911b8ae455fb6df71f32e237a58` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:58.511073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] RBAC: Authorizing `identity:get_user(user_id=7f709911b8ae455fb6df71f32e237a58)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:58.512501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:58.514151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 567/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:00:58 2022] GET /identity/v3/users/7f709911b8ae455fb6df71f32e237a58 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:58.519904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:58.554241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:58.619591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf63975-21c2-454e-941b-2ceeacf0dbaa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:58.652131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:58.652774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:58.653056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:58.653334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:58.719456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:58.720925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:58.755039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:58.819343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 561/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:00:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:00:58.825432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:58.899007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:58.923094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98161bcc-b097-4472-897c-65d87fd08441 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:58.956622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a56af-ba70-45c1-8270-526b4f137be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:58.957523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a56af-ba70-45c1-8270-526b4f137be2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:58.957838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a56af-ba70-45c1-8270-526b4f137be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:58.958122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a56af-ba70-45c1-8270-526b4f137be2 None admin] PATH_INFO: `/v3/projects/713479262ec84f0f86df3bbb9c1dc822` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:58.959569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a56af-ba70-45c1-8270-526b4f137be2 None admin] RBAC: Authorizing `identity:get_project(project_id=713479262ec84f0f86df3bbb9c1dc822)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:58.961122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a56af-ba70-45c1-8270-526b4f137be2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:58.962840 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 568/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:00:58 2022] GET /identity/v3/projects/713479262ec84f0f86df3bbb9c1dc822 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:00:58.969326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:00:59.068080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439bfdb-3c47-472a-9184-fc307f0cc567 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:00:59.101843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830c362-2f98-40f6-a9ef-d7e4ea8f1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:00:59.102493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830c362-2f98-40f6-a9ef-d7e4ea8f1bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:00:59.102781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830c362-2f98-40f6-a9ef-d7e4ea8f1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:00:59.103063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830c362-2f98-40f6-a9ef-d7e4ea8f1bb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:00:59.103887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830c362-2f98-40f6-a9ef-d7e4ea8f1bb7 None admin] RBAC: Adding query filter params (role.id=b59592ce5fcf4e3981fd342b5352ee2b, scope.project.id=713479262ec84f0f86df3bbb9c1dc822, user.id=7f709911b8ae455fb6df71f32e237a58) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:00:59.104205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830c362-2f98-40f6-a9ef-d7e4ea8f1bb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:00:59.105646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830c362-2f98-40f6-a9ef-d7e4ea8f1bb7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:00:59.107476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:00:59.112985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 562/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:00:58 2022] GET /identity/v3/role_assignments?user.id=7f709911b8ae455fb6df71f32e237a58&scope.project.id=713479262ec84f0f86df3bbb9c1dc822&role.id=b59592ce5fcf4e3981fd342b5352ee2b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:01.248713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d64dd-ddd2-4cff-b383-cb85a91199d0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:01.249235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d64dd-ddd2-4cff-b383-cb85a91199d0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:01.249362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d64dd-ddd2-4cff-b383-cb85a91199d0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:01.250787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 569/1131] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:01.256104 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c177856-c6c8-4701-bf17-f0b4655eb84b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:01.256489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c177856-c6c8-4701-bf17-f0b4655eb84b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:01.256669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c177856-c6c8-4701-bf17-f0b4655eb84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:01.273374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5c177856-c6c8-4701-bf17-f0b4655eb84b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:01.274559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c177856-c6c8-4701-bf17-f0b4655eb84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:01.339783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 563/1132] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:01.364490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d64dd-ddd2-4cff-b383-cb85a91199d0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:01.462632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d64dd-ddd2-4cff-b383-cb85a91199d0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:01.495855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb5b39-a010-4587-a468-d0f3c4fbf544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:01.496631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb5b39-a010-4587-a468-d0f3c4fbf544 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:01.496910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb5b39-a010-4587-a468-d0f3c4fbf544 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:01.497182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb5b39-a010-4587-a468-d0f3c4fbf544 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:01.498341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb5b39-a010-4587-a468-d0f3c4fbf544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:01.499797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb5b39-a010-4587-a468-d0f3c4fbf544 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:01.501379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 570/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:01.507201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c177856-c6c8-4701-bf17-f0b4655eb84b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:01.602719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c177856-c6c8-4701-bf17-f0b4655eb84b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:01.627614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:01.635904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982d588-fbb6-4a16-93c4-e4ed659a7ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:01.636556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d588-fbb6-4a16-93c4-e4ed659a7ce3 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:01.636839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d588-fbb6-4a16-93c4-e4ed659a7ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:01.637117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d588-fbb6-4a16-93c4-e4ed659a7ce3 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:01.637843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982d588-fbb6-4a16-93c4-e4ed659a7ce3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:01.639276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982d588-fbb6-4a16-93c4-e4ed659a7ce3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:01.663443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 564/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:01:01 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:01:03.794996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4701-53e9-440c-b215-3dffdf0b1c3f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:03.795547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4701-53e9-440c-b215-3dffdf0b1c3f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:03.795547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a4701-53e9-440c-b215-3dffdf0b1c3f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:03.796921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 571/1135] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:03.802786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f408625-7229-4063-9c88-83f51b4de859 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:03.803182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f408625-7229-4063-9c88-83f51b4de859 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:03.803359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f408625-7229-4063-9c88-83f51b4de859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:03.819739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3f408625-7229-4063-9c88-83f51b4de859 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:03.820911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f408625-7229-4063-9c88-83f51b4de859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:03.885671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 565/1136] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:03.909819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a4701-53e9-440c-b215-3dffdf0b1c3f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:03.922499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:04.009782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a4701-53e9-440c-b215-3dffdf0b1c3f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:04.044326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664613f-f665-4735-b5dc-715688ee266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:04.045163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664613f-f665-4735-b5dc-715688ee266c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:04.045385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664613f-f665-4735-b5dc-715688ee266c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:04.045657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664613f-f665-4735-b5dc-715688ee266c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:04.046864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664613f-f665-4735-b5dc-715688ee266c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:04.048429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664613f-f665-4735-b5dc-715688ee266c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:04.050060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 572/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:04.056323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f408625-7229-4063-9c88-83f51b4de859 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:04.153420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f408625-7229-4063-9c88-83f51b4de859 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:04.164455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:04.188342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0b731-0f64-42c9-a9a4-885e07f87432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:04.189011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0b731-0f64-42c9-a9a4-885e07f87432 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:04.189296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0b731-0f64-42c9-a9a4-885e07f87432 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:04.189575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0b731-0f64-42c9-a9a4-885e07f87432 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:04.190313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0b731-0f64-42c9-a9a4-885e07f87432 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:04.191823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0b731-0f64-42c9-a9a4-885e07f87432 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:04.211897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 566/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 16:01:04 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:01:06.356311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65333-1b60-4a6c-9de5-7fef738c36be None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:06.356834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65333-1b60-4a6c-9de5-7fef738c36be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:06.356834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65333-1b60-4a6c-9de5-7fef738c36be None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:06.358258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 573/1139] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:06.364066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ef52f-4037-4a68-b513-d6d882667e4c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:06.364466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ef52f-4037-4a68-b513-d6d882667e4c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:06.364622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ef52f-4037-4a68-b513-d6d882667e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:06.382245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-304ef52f-4037-4a68-b513-d6d882667e4c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:06.383410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-304ef52f-4037-4a68-b513-d6d882667e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:06.448279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 567/1140] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:06.471357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65333-1b60-4a6c-9de5-7fef738c36be None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:06.548811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:06.570871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65333-1b60-4a6c-9de5-7fef738c36be None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:06.603990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:06.604771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:06.605051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:06.605325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:06.615025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:06.622442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:06.623528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:01:06.624523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 574/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:06.630208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ef52f-4037-4a68-b513-d6d882667e4c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:06.721094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:06.728380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ef52f-4037-4a68-b513-d6d882667e4c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:06.763206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:06.763862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:06.764146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:06.764422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:06.765160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:06.765467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:06.766873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:06.775004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 568/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:06.781023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:06.878670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382ec8a-1582-40d6-8e38-20462a48745f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:06.911743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:06.912600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:06.912883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:06.913163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] PATH_INFO: `/v3/users/7fed9154a0a2430b8f8ae0eab16b7d6f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:06.920710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:06.927083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] RBAC: Authorizing `identity:get_user(user_id=7fed9154a0a2430b8f8ae0eab16b7d6f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:06.928530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:06.930230 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 575/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:01:06 2022] GET /identity/v3/users/7fed9154a0a2430b8f8ae0eab16b7d6f => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:06.936001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:07.038847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa394a-73d5-4a7c-baeb-5f6b7793448d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:07.072038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:07.072684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:07.072971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:07.073252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:07.100497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:07.140462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:07.141961 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:07.238196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 569/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:07.244176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:07.340925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae2459-3224-4bb4-86c0-7305b58e2613 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:07.374209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234a905-f3ff-428b-83aa-f6ea8c97157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:07.375049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a905-f3ff-428b-83aa-f6ea8c97157e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:07.375328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a905-f3ff-428b-83aa-f6ea8c97157e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:07.375599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a905-f3ff-428b-83aa-f6ea8c97157e None admin] PATH_INFO: `/v3/projects/651803b1dd7f4faba042d48f0bb81b30` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:07.376769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234a905-f3ff-428b-83aa-f6ea8c97157e None admin] RBAC: Authorizing `identity:get_project(project_id=651803b1dd7f4faba042d48f0bb81b30)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:07.378220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234a905-f3ff-428b-83aa-f6ea8c97157e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:07.380418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 576/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:01:07 2022] GET /identity/v3/projects/651803b1dd7f4faba042d48f0bb81b30 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:07.386505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:07.443133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:07.483661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3398d-5c2f-4bbd-b8ee-1b44112a8e1d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:07.516816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bab8a0-b36c-444a-b5e7-9bb09ed4c1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:07.517460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bab8a0-b36c-444a-b5e7-9bb09ed4c1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:07.517817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bab8a0-b36c-444a-b5e7-9bb09ed4c1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:07.518051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bab8a0-b36c-444a-b5e7-9bb09ed4c1d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:07.518867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bab8a0-b36c-444a-b5e7-9bb09ed4c1d1 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=651803b1dd7f4faba042d48f0bb81b30, user.id=7fed9154a0a2430b8f8ae0eab16b7d6f) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:07.519234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bab8a0-b36c-444a-b5e7-9bb09ed4c1d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:07.521163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bab8a0-b36c-444a-b5e7-9bb09ed4c1d1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:07.526530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 570/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:01:07 2022] GET /identity/v3/role_assignments?user.id=7fed9154a0a2430b8f8ae0eab16b7d6f&scope.project.id=651803b1dd7f4faba042d48f0bb81b30&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:09.613373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394c424-e265-4f3b-bc67-994d8a3f2b9e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:09.614044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394c424-e265-4f3b-bc67-994d8a3f2b9e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:09.614044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394c424-e265-4f3b-bc67-994d8a3f2b9e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:09.615329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 577/1147] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:09.620542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29c3b7-53b7-486d-9b17-04532ea9161b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:09.620917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29c3b7-53b7-486d-9b17-04532ea9161b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:09.621107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29c3b7-53b7-486d-9b17-04532ea9161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:09.637844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-de29c3b7-53b7-486d-9b17-04532ea9161b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:09.639014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-de29c3b7-53b7-486d-9b17-04532ea9161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:09.703796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 571/1148] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:09.727250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394c424-e265-4f3b-bc67-994d8a3f2b9e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:09.788595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:09.825757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394c424-e265-4f3b-bc67-994d8a3f2b9e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:09.858907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:09.859678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:09.859959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:09.860234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:09.869985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:09.877694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:09.879311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:01:09.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 578/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:09.885478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29c3b7-53b7-486d-9b17-04532ea9161b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:09.936358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:09.982218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de29c3b7-53b7-486d-9b17-04532ea9161b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:10.015839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:10.016502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:10.016787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:10.017067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:10.017839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:10.018150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:10.019580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:10.027447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 572/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:10.033338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:10.132244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9d1ea-5b27-452f-b041-ba777cd6643c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:10.155975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:10.167089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:10.167953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:10.168236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:10.168511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] PATH_INFO: `/v3/users/7fed9154a0a2430b8f8ae0eab16b7d6f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:10.169683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] RBAC: Authorizing `identity:get_user(user_id=7fed9154a0a2430b8f8ae0eab16b7d6f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:10.171139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:10.172754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 579/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:01:10 2022] GET /identity/v3/users/7fed9154a0a2430b8f8ae0eab16b7d6f => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:10.179057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:10.277768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c155af-6e5d-4110-994a-b8a26e90a2c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:10.290919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:10.310580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d73ef-50c5-46e0-b7c9-4dcbc5c68c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:10.311440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d73ef-50c5-46e0-b7c9-4dcbc5c68c91 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:10.311719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d73ef-50c5-46e0-b7c9-4dcbc5c68c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:10.311992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d73ef-50c5-46e0-b7c9-4dcbc5c68c91 None admin] PATH_INFO: `/v3/projects/651803b1dd7f4faba042d48f0bb81b30` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:10.313156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d73ef-50c5-46e0-b7c9-4dcbc5c68c91 None admin] RBAC: Authorizing `identity:get_project(project_id=651803b1dd7f4faba042d48f0bb81b30)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:10.314623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d73ef-50c5-46e0-b7c9-4dcbc5c68c91 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:10.316853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 573/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:01:10 2022] GET /identity/v3/projects/651803b1dd7f4faba042d48f0bb81b30 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:10.322801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:10.420341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65bf76-f7b0-4ade-9478-7c5f9240db46 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:10.453062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3372d65-07ed-49e0-a17e-338bde2e198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:10.453972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372d65-07ed-49e0-a17e-338bde2e198b None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:10.454256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372d65-07ed-49e0-a17e-338bde2e198b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:10.454539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372d65-07ed-49e0-a17e-338bde2e198b None admin] PATH_INFO: `/v3/projects/651803b1dd7f4faba042d48f0bb81b30/users/7fed9154a0a2430b8f8ae0eab16b7d6f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:10.456465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3372d65-07ed-49e0-a17e-338bde2e198b None admin] RBAC: Authorizing `identity:create_grant(project_id=651803b1dd7f4faba042d48f0bb81b30, user_id=7fed9154a0a2430b8f8ae0eab16b7d6f, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:10.457941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3372d65-07ed-49e0-a17e-338bde2e198b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:10.471891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 580/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:01:10 2022] PUT /identity/v3/projects/651803b1dd7f4faba042d48f0bb81b30/users/7fed9154a0a2430b8f8ae0eab16b7d6f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:01:12.569742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db922421-e279-4b8f-98c8-1db53f140307 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:12.570294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db922421-e279-4b8f-98c8-1db53f140307 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:12.570294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db922421-e279-4b8f-98c8-1db53f140307 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:12.571689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 574/1154] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:12.576811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e1559-16ab-468f-a7de-3466faeb4c72 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:12.577228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e1559-16ab-468f-a7de-3466faeb4c72 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:12.577375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e1559-16ab-468f-a7de-3466faeb4c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:12.594523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-245e1559-16ab-468f-a7de-3466faeb4c72 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:12.595703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-245e1559-16ab-468f-a7de-3466faeb4c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:12.641834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:12.662388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 581/1155] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:12.685707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db922421-e279-4b8f-98c8-1db53f140307 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:12.786148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db922421-e279-4b8f-98c8-1db53f140307 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:12.818225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:12.819018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:12.819294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:12.819565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:12.830069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:12.838025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:12.839282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:01:12.840595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 575/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:12.845592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e1559-16ab-468f-a7de-3466faeb4c72 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:12.944117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e1559-16ab-468f-a7de-3466faeb4c72 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:12.977917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:12.978576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:12.978859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:12.979134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:12.979863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:12.980165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:12.981519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:12.989298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 582/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:12.995205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:13.039204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:13.095746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bf80f-593a-48bd-8632-ac46ff8b8b0d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:13.129916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:13.130794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:13.131088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:13.131370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] PATH_INFO: `/v3/users/7fed9154a0a2430b8f8ae0eab16b7d6f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:13.132592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] RBAC: Authorizing `identity:get_user(user_id=7fed9154a0a2430b8f8ae0eab16b7d6f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:13.134080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:13.135736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 576/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:01:12 2022] GET /identity/v3/users/7fed9154a0a2430b8f8ae0eab16b7d6f => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:13.141413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:13.189513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:13.243392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52dcd7-1c36-4097-a34f-e5e17b6c0830 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:13.276301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:13.276943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:13.277226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:13.277504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:13.342899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:13.344329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:13.386659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:13.439393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 583/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:13.445843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:13.543045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a771c-2bc7-4e32-8adc-333fd781d43a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:13.553869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:13.577810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8fd93-a435-4bc7-9b48-9a975e375054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:13.578661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8fd93-a435-4bc7-9b48-9a975e375054 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:13.578944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8fd93-a435-4bc7-9b48-9a975e375054 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:13.579219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8fd93-a435-4bc7-9b48-9a975e375054 None admin] PATH_INFO: `/v3/projects/651803b1dd7f4faba042d48f0bb81b30` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:13.580391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8fd93-a435-4bc7-9b48-9a975e375054 None admin] RBAC: Authorizing `identity:get_project(project_id=651803b1dd7f4faba042d48f0bb81b30)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:13.582427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d8fd93-a435-4bc7-9b48-9a975e375054 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:13.584145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 577/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:01:13 2022] GET /identity/v3/projects/651803b1dd7f4faba042d48f0bb81b30 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:13.590153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:13.686027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076566a4-1f21-478b-a60a-18213490b816 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:13.720026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea64d5b-6a5b-4fad-b0da-54f95b628680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:13.720665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64d5b-6a5b-4fad-b0da-54f95b628680 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:13.720965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64d5b-6a5b-4fad-b0da-54f95b628680 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:13.721248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64d5b-6a5b-4fad-b0da-54f95b628680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:13.722164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea64d5b-6a5b-4fad-b0da-54f95b628680 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=651803b1dd7f4faba042d48f0bb81b30, user.id=7fed9154a0a2430b8f8ae0eab16b7d6f) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:13.722968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea64d5b-6a5b-4fad-b0da-54f95b628680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:13.724380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea64d5b-6a5b-4fad-b0da-54f95b628680 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:13.730050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 584/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:01:13 2022] GET /identity/v3/role_assignments?user.id=7fed9154a0a2430b8f8ae0eab16b7d6f&scope.project.id=651803b1dd7f4faba042d48f0bb81b30&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:15.859021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded1c3-1bb5-40a6-8401-af88697997f6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:15.859590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded1c3-1bb5-40a6-8401-af88697997f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:15.859590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ded1c3-1bb5-40a6-8401-af88697997f6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:15.860935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 578/1162] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:15.866542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1d99-36d7-46e7-903f-f21651c51182 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:15.866965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1d99-36d7-46e7-903f-f21651c51182 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:15.867107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e1d99-36d7-46e7-903f-f21651c51182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:15.883757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e17e1d99-36d7-46e7-903f-f21651c51182 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:15.884927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e17e1d99-36d7-46e7-903f-f21651c51182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:15.886857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:15.949639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 585/1163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:15.974161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ded1c3-1bb5-40a6-8401-af88697997f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:16.073923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ded1c3-1bb5-40a6-8401-af88697997f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:16.107376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:16.108069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:16.108406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:16.108685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:16.153958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:16.176498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:16.177981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:16.273231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 579/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:16.279106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e1d99-36d7-46e7-903f-f21651c51182 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:16.373100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e1d99-36d7-46e7-903f-f21651c51182 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:16.406880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b157c78-741a-4fa4-b266-84f245bf7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:16.407592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b157c78-741a-4fa4-b266-84f245bf7b29 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:16.407873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b157c78-741a-4fa4-b266-84f245bf7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:16.408147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b157c78-741a-4fa4-b266-84f245bf7b29 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:16.415079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b157c78-741a-4fa4-b266-84f245bf7b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:16.416501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b157c78-741a-4fa4-b266-84f245bf7b29 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:16.418159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0b157c78-741a-4fa4-b266-84f245bf7b29 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:16.431358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:16.432648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:01:16.433776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 586/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:01:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:16.438625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:16.514515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:16.536840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9061b9-c7eb-492d-8a43-7b590c19a2ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:16.570097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770f644-1e85-438f-b8cc-6d3b919ce3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:16.570756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770f644-1e85-438f-b8cc-6d3b919ce3cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:16.571040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770f644-1e85-438f-b8cc-6d3b919ce3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:16.571318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770f644-1e85-438f-b8cc-6d3b919ce3cc None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:16.572028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770f644-1e85-438f-b8cc-6d3b919ce3cc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:16.572335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770f644-1e85-438f-b8cc-6d3b919ce3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:16.573710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770f644-1e85-438f-b8cc-6d3b919ce3cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:16.580783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 580/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Thu Jun 16 16:01:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:18.668055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81c8d9-630f-431b-b22d-1a885e07b13d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:18.668632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81c8d9-630f-431b-b22d-1a885e07b13d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:18.668632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81c8d9-630f-431b-b22d-1a885e07b13d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:18.670014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 587/1167] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:18.675052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c43e95-f8ed-4754-8473-2b09df72d353 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:18.675454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c43e95-f8ed-4754-8473-2b09df72d353 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:18.675607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c43e95-f8ed-4754-8473-2b09df72d353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:18.692311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-34c43e95-f8ed-4754-8473-2b09df72d353 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:18.693469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34c43e95-f8ed-4754-8473-2b09df72d353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:18.758358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 581/1168] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:18.781375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81c8d9-630f-431b-b22d-1a885e07b13d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:18.880059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81c8d9-630f-431b-b22d-1a885e07b13d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:18.914805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccac97-2a06-44ae-b9e1-3293fbf542c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:18.915466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccac97-2a06-44ae-b9e1-3293fbf542c6 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:18.915749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccac97-2a06-44ae-b9e1-3293fbf542c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:18.916031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccac97-2a06-44ae-b9e1-3293fbf542c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:18.916767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccac97-2a06-44ae-b9e1-3293fbf542c6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:18.918224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfccac97-2a06-44ae-b9e1-3293fbf542c6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:18.933456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 588/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Thu Jun 16 16:01:18 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:01:21.064971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e733eb5-e4ee-4ab9-93fa-cb0644eb4974 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:21.065513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e733eb5-e4ee-4ab9-93fa-cb0644eb4974 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:21.065513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e733eb5-e4ee-4ab9-93fa-cb0644eb4974 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:21.067057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 582/1170] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:21.072438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85d85-b5f5-46f1-ba7c-99490d9e29cc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:21.072885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85d85-b5f5-46f1-ba7c-99490d9e29cc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:21.072991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd85d85-b5f5-46f1-ba7c-99490d9e29cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:21.089318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5dd85d85-b5f5-46f1-ba7c-99490d9e29cc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:21.090534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5dd85d85-b5f5-46f1-ba7c-99490d9e29cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:21.143306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:21.157551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 589/1171] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:21.181745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e733eb5-e4ee-4ab9-93fa-cb0644eb4974 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:21.251663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:21.281612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e733eb5-e4ee-4ab9-93fa-cb0644eb4974 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:21.314257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e166d7-7703-4cbd-85a8-d3af332a3a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:21.315154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e166d7-7703-4cbd-85a8-d3af332a3a5a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:21.315386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e166d7-7703-4cbd-85a8-d3af332a3a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:21.315687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e166d7-7703-4cbd-85a8-d3af332a3a5a None admin] PATH_INFO: `/v3/domains/b500bd8835884cbcbb6e55d94b6a25f0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:21.327140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e166d7-7703-4cbd-85a8-d3af332a3a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=b500bd8835884cbcbb6e55d94b6a25f0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:21.329179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e166d7-7703-4cbd-85a8-d3af332a3a5a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:21.332375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 583/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Thu Jun 16 16:01:21 2022] GET /identity/v3/domains/b500bd8835884cbcbb6e55d94b6a25f0 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:21.338223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85d85-b5f5-46f1-ba7c-99490d9e29cc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:21.435731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd85d85-b5f5-46f1-ba7c-99490d9e29cc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:21.470269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592fb10-55e9-4b50-ab12-cc15381917b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:21.470922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592fb10-55e9-4b50-ab12-cc15381917b7 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:21.471206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592fb10-55e9-4b50-ab12-cc15381917b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:21.471482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592fb10-55e9-4b50-ab12-cc15381917b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:21.472749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592fb10-55e9-4b50-ab12-cc15381917b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:21.474210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592fb10-55e9-4b50-ab12-cc15381917b7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:21.499222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 590/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:01:21 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:01:23.645596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634a634-c11c-4d46-abe5-b1a3dedbae9e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:23.646141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634a634-c11c-4d46-abe5-b1a3dedbae9e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:23.646243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634a634-c11c-4d46-abe5-b1a3dedbae9e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:23.647561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 584/1174] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:23.654789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04409fbd-ee65-4fe2-a150-8db2dc5f01a7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:23.654789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04409fbd-ee65-4fe2-a150-8db2dc5f01a7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:23.654789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04409fbd-ee65-4fe2-a150-8db2dc5f01a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:23.672573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-04409fbd-ee65-4fe2-a150-8db2dc5f01a7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:23.673838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-04409fbd-ee65-4fe2-a150-8db2dc5f01a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:23.708167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:23.743026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 591/1175] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:23.768506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634a634-c11c-4d46-abe5-b1a3dedbae9e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:23.868920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634a634-c11c-4d46-abe5-b1a3dedbae9e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:23.897858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:23.903116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2ecab-d5ff-4a3a-a272-54c4fb337ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:23.903978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ecab-d5ff-4a3a-a272-54c4fb337ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:23.904266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ecab-d5ff-4a3a-a272-54c4fb337ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:23.904546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ecab-d5ff-4a3a-a272-54c4fb337ca4 None admin] PATH_INFO: `/v3/domains/b500bd8835884cbcbb6e55d94b6a25f0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:23.905734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2ecab-d5ff-4a3a-a272-54c4fb337ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b500bd8835884cbcbb6e55d94b6a25f0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:23.907176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2ecab-d5ff-4a3a-a272-54c4fb337ca4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:23.908799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 585/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Thu Jun 16 16:01:23 2022] GET /identity/v3/domains/b500bd8835884cbcbb6e55d94b6a25f0 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:23.914568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04409fbd-ee65-4fe2-a150-8db2dc5f01a7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:24.009665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04409fbd-ee65-4fe2-a150-8db2dc5f01a7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:24.042166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e1417-aca5-49e9-bb1d-4aa6ecac99bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:24.042821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e1417-aca5-49e9-bb1d-4aa6ecac99bb None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:24.043104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e1417-aca5-49e9-bb1d-4aa6ecac99bb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:24.043381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e1417-aca5-49e9-bb1d-4aa6ecac99bb None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:24.044076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e1417-aca5-49e9-bb1d-4aa6ecac99bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:24.045505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e1417-aca5-49e9-bb1d-4aa6ecac99bb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:24.064277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 592/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 16:01:23 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:01:26.201146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05092-53b6-4ee5-837f-d2ac6f84f5f6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:26.201787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05092-53b6-4ee5-837f-d2ac6f84f5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:26.201787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e05092-53b6-4ee5-837f-d2ac6f84f5f6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:26.203107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 586/1178] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:26.208743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0afc0-c7b5-4745-9f08-48522664d03c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:26.209151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0afc0-c7b5-4745-9f08-48522664d03c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:26.209306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0afc0-c7b5-4745-9f08-48522664d03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:26.226620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-95c0afc0-c7b5-4745-9f08-48522664d03c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:26.227803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-95c0afc0-c7b5-4745-9f08-48522664d03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:26.247807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:26.295307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 593/1179] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:26.319188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e05092-53b6-4ee5-837f-d2ac6f84f5f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:26.418941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e05092-53b6-4ee5-837f-d2ac6f84f5f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:26.450451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:26.451231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:26.451586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:26.451857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:26.462518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:26.470371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:26.471511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:01:26.472990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 587/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:26.478090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0afc0-c7b5-4745-9f08-48522664d03c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:26.573326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c0afc0-c7b5-4745-9f08-48522664d03c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:26.606654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:26.607312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:26.607601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:26.607880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:26.608616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:26.608922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:26.610319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:26.618676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 594/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:26.624560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:26.682216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:26.724124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c7099-7cc7-4b65-85f7-80b734232141 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:26.758116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:26.758979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:26.759262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:26.759538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] PATH_INFO: `/v3/users/57b3227b158949788bc4533f29b641e7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:26.772217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] RBAC: Authorizing `identity:get_user(user_id=57b3227b158949788bc4533f29b641e7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:26.773658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:26.775335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 588/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:01:26 2022] GET /identity/v3/users/57b3227b158949788bc4533f29b641e7 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:26.781098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:26.801056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:26.879195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30935a-ea7c-48dc-a0c4-9a6040bdf43c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:26.912417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:26.913056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:26.913342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:26.913664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:26.979092 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:26.980443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:26.998335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:27.085833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 595/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:27.091728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:27.189016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0601d-06a2-45c3-9da5-198f05f0a730 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:27.205904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:27.223498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d3766-f79c-4d43-bfd0-c38afce1dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:27.224352 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3766-f79c-4d43-bfd0-c38afce1dc78 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:27.224629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3766-f79c-4d43-bfd0-c38afce1dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:27.224899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d3766-f79c-4d43-bfd0-c38afce1dc78 None admin] PATH_INFO: `/v3/projects/e341818b07f643f4abb7b55975e0740c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:27.226084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d3766-f79c-4d43-bfd0-c38afce1dc78 None admin] RBAC: Authorizing `identity:get_project(project_id=e341818b07f643f4abb7b55975e0740c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:27.227511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d3766-f79c-4d43-bfd0-c38afce1dc78 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:27.229234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 589/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:01:27 2022] GET /identity/v3/projects/e341818b07f643f4abb7b55975e0740c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:27.235322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:27.331682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009a419-f17c-44b5-8afc-147506fbc60c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:27.348361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:27.365243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4aee2-be47-475b-8e29-997b09d362ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:27.365919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4aee2-be47-475b-8e29-997b09d362ad None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:27.366207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4aee2-be47-475b-8e29-997b09d362ad None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:27.366485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4aee2-be47-475b-8e29-997b09d362ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:27.367309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4aee2-be47-475b-8e29-997b09d362ad None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=e341818b07f643f4abb7b55975e0740c, user.id=57b3227b158949788bc4533f29b641e7) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:27.367622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4aee2-be47-475b-8e29-997b09d362ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:27.369055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4aee2-be47-475b-8e29-997b09d362ad None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:27.375388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 596/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:01:27 2022] GET /identity/v3/role_assignments?user.id=57b3227b158949788bc4533f29b641e7&scope.project.id=e341818b07f643f4abb7b55975e0740c&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:29.461492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367627e-2d30-4cfc-a2f4-16182522e45a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:29.462034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367627e-2d30-4cfc-a2f4-16182522e45a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:29.462034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367627e-2d30-4cfc-a2f4-16182522e45a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:29.463371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 590/1186] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:29.468741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9fbca-1541-456e-8d3b-71b77ed4813d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:29.469138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9fbca-1541-456e-8d3b-71b77ed4813d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:29.469297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9fbca-1541-456e-8d3b-71b77ed4813d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:29.486113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5ec9fbca-1541-456e-8d3b-71b77ed4813d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:29.487351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5ec9fbca-1541-456e-8d3b-71b77ed4813d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:29.551182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 597/1187] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:29.573447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367627e-2d30-4cfc-a2f4-16182522e45a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:29.672746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367627e-2d30-4cfc-a2f4-16182522e45a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:29.706890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:29.707671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:29.707953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:29.708226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:29.720185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:29.728776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:29.729979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:01:29.731420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 591/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:01:29.737093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9fbca-1541-456e-8d3b-71b77ed4813d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:29.836819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec9fbca-1541-456e-8d3b-71b77ed4813d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:29.851036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:29.875503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:29.876177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:29.876460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:29.876738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:29.877477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:29.878345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:29.879716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:29.888558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 598/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:29.895133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:29.942386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:29.997501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd45310-b2b3-4409-9597-433ae33804f3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:30.030969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:30.031833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:30.032114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:30.032387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] PATH_INFO: `/v3/users/57b3227b158949788bc4533f29b641e7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:30.033550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] RBAC: Authorizing `identity:get_user(user_id=57b3227b158949788bc4533f29b641e7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:30.035502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:30.037125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 592/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:01:29 2022] GET /identity/v3/users/57b3227b158949788bc4533f29b641e7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:30.042903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:30.141443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f12b69-ddaf-4c32-93d9-56be983806ac None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:30.174495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3a4f2-3f25-45cf-8533-1f08141c83d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:30.175942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3a4f2-3f25-45cf-8533-1f08141c83d0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:30.176219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3a4f2-3f25-45cf-8533-1f08141c83d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:30.176488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3a4f2-3f25-45cf-8533-1f08141c83d0 None admin] PATH_INFO: `/v3/projects/e341818b07f643f4abb7b55975e0740c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:30.177643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3a4f2-3f25-45cf-8533-1f08141c83d0 None admin] RBAC: Authorizing `identity:get_project(project_id=e341818b07f643f4abb7b55975e0740c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:30.179087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3a4f2-3f25-45cf-8533-1f08141c83d0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:30.180758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 599/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:01:30 2022] GET /identity/v3/projects/e341818b07f643f4abb7b55975e0740c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:30.186669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:30.287160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5314e-0f83-4c6a-9908-3fe422b73718 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:30.301957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:30.322216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68595b36-98b9-417e-87ac-e79edfe7d09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:30.323099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595b36-98b9-417e-87ac-e79edfe7d09b None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:30.323379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595b36-98b9-417e-87ac-e79edfe7d09b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:30.323656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595b36-98b9-417e-87ac-e79edfe7d09b None admin] PATH_INFO: `/v3/projects/e341818b07f643f4abb7b55975e0740c/users/57b3227b158949788bc4533f29b641e7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:30.325838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68595b36-98b9-417e-87ac-e79edfe7d09b None admin] RBAC: Authorizing `identity:create_grant(project_id=e341818b07f643f4abb7b55975e0740c, user_id=57b3227b158949788bc4533f29b641e7, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:30.327245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68595b36-98b9-417e-87ac-e79edfe7d09b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:30.344596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 593/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:01:30 2022] PUT /identity/v3/projects/e341818b07f643f4abb7b55975e0740c/users/57b3227b158949788bc4533f29b641e7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:01:32.432782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae850c1-8e61-4ded-a7ea-27964944c389 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:32.433315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae850c1-8e61-4ded-a7ea-27964944c389 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:32.433315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae850c1-8e61-4ded-a7ea-27964944c389 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:32.434633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 600/1193] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:01:32.439886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b082a-6d64-4932-a9cd-43ce0cef03b6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:32.440310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b082a-6d64-4932-a9cd-43ce0cef03b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:32.440443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b082a-6d64-4932-a9cd-43ce0cef03b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:32.467169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e03b082a-6d64-4932-a9cd-43ce0cef03b6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:01:32.468629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e03b082a-6d64-4932-a9cd-43ce0cef03b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:01:32.541277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 594/1194] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:01:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:01:32.564452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae850c1-8e61-4ded-a7ea-27964944c389 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:32.611216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:32.671374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae850c1-8e61-4ded-a7ea-27964944c389 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:32.707332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:32.708043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:32.708326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:32.708598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:32.723072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:01:32.730640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:01:32.731752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:01:32.732638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 601/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:01:32.738554 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b082a-6d64-4932-a9cd-43ce0cef03b6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:32.840865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b082a-6d64-4932-a9cd-43ce0cef03b6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:32.843475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:32.876505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:32.877174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:32.877460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:32.877796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:32.878551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:32.878867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:32.880262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:32.889663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 595/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:32.895815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:32.999813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520569df-cea9-4bb4-91b5-d48167d88bd8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:33.001444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:33.040032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:33.040919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:33.041206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:33.041485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] PATH_INFO: `/v3/users/57b3227b158949788bc4533f29b641e7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:33.042785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] RBAC: Authorizing `identity:get_user(user_id=57b3227b158949788bc4533f29b641e7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:33.044246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:33.046001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 602/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu Jun 16 16:01:32 2022] GET /identity/v3/users/57b3227b158949788bc4533f29b641e7 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:33.052032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:33.161333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc1a2-fd45-41f9-9b50-6e85a3470aec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:33.198899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:33.199559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:33.199845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:33.200128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:33.235671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:33.275447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:33.276928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:33.381113 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 596/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:01:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:01:33.387193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:33.487619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea2d99-f6e9-4d50-9838-27660ad1f9d9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:33.520400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b0d48-406d-42b0-b366-e97a9975f145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:33.521244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b0d48-406d-42b0-b366-e97a9975f145 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:33.521521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b0d48-406d-42b0-b366-e97a9975f145 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:33.521819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b0d48-406d-42b0-b366-e97a9975f145 None admin] PATH_INFO: `/v3/projects/e341818b07f643f4abb7b55975e0740c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:33.522969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b0d48-406d-42b0-b366-e97a9975f145 None admin] RBAC: Authorizing `identity:get_project(project_id=e341818b07f643f4abb7b55975e0740c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:33.524391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b0d48-406d-42b0-b366-e97a9975f145 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:33.526154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 603/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Thu Jun 16 16:01:33 2022] GET /identity/v3/projects/e341818b07f643f4abb7b55975e0740c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:33.532206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:01:33.597448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:33.631167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedfb83-ccd8-4f02-883d-cb62e21c0583 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:01:33.663710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb8a8-9323-4ee1-b387-71c54e8a6327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:01:33.664350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb8a8-9323-4ee1-b387-71c54e8a6327 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:01:33.664629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb8a8-9323-4ee1-b387-71c54e8a6327 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:01:33.664900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb8a8-9323-4ee1-b387-71c54e8a6327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:01:33.665741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcb8a8-9323-4ee1-b387-71c54e8a6327 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=e341818b07f643f4abb7b55975e0740c, user.id=57b3227b158949788bc4533f29b641e7) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:01:33.666054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcb8a8-9323-4ee1-b387-71c54e8a6327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:01:33.667485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcb8a8-9323-4ee1-b387-71c54e8a6327 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:01:33.673238 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 597/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:01:33 2022] GET /identity/v3/role_assignments?user.id=57b3227b158949788bc4533f29b641e7&scope.project.id=e341818b07f643f4abb7b55975e0740c&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:01:43.515999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=81 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:01:43.670660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=35 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:30.467715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a901023-d209-4887-9924-f8a492a50035 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:30.467715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a901023-d209-4887-9924-f8a492a50035 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:30.467715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a901023-d209-4887-9924-f8a492a50035 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:30.468832 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 604/1201] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:30.473861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a310c9-8365-4d16-b989-87be7c84e49d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:30.474307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a310c9-8365-4d16-b989-87be7c84e49d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:30.474418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a310c9-8365-4d16-b989-87be7c84e49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:30.493664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a0a310c9-8365-4d16-b989-87be7c84e49d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:30.495139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0a310c9-8365-4d16-b989-87be7c84e49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:30.598132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 598/1202] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:30.626172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a901023-d209-4887-9924-f8a492a50035 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:30.757117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a901023-d209-4887-9924-f8a492a50035 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:30.796675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:30.797562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:30.797924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:30.798208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:30.812849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:30.815236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:30.835640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:30.836806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:30.837838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:30.837838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:30.837838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:30.837838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:30.837838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 605/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:30.843840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a310c9-8365-4d16-b989-87be7c84e49d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:30.979050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a310c9-8365-4d16-b989-87be7c84e49d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:31.019421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:31.028082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d9b26-2af5-45f8-8a2b-7ae4dac0eb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:31.028795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d9b26-2af5-45f8-8a2b-7ae4dac0eb7f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:31.029082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d9b26-2af5-45f8-8a2b-7ae4dac0eb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:31.029360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d9b26-2af5-45f8-8a2b-7ae4dac0eb7f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:31.030175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d9b26-2af5-45f8-8a2b-7ae4dac0eb7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:31.030447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d9b26-2af5-45f8-8a2b-7ae4dac0eb7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:31.032220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184d9b26-2af5-45f8-8a2b-7ae4dac0eb7f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:31.046256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 599/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:31.053450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:31.114285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:31.182393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab2ba1-31ee-4f6b-b64e-37e0e3e1e260 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:31.238558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce02ab-c8d4-4c52-8981-8bfaf580011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:31.239272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce02ab-c8d4-4c52-8981-8bfaf580011e None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:31.239558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce02ab-c8d4-4c52-8981-8bfaf580011e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:31.239838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce02ab-c8d4-4c52-8981-8bfaf580011e None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:31.240548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce02ab-c8d4-4c52-8981-8bfaf580011e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:31.242142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce02ab-c8d4-4c52-8981-8bfaf580011e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:31.267588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 606/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Thu Jun 16 16:02:31 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:02:33.612046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80ad82-40f6-4164-9c4d-195b1db56f30 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:33.612604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80ad82-40f6-4164-9c4d-195b1db56f30 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:33.612604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80ad82-40f6-4164-9c4d-195b1db56f30 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:33.614290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 600/1206] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:02:33.620056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3bd7-c956-43db-b41c-ef482069e0ec None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:33.620565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3bd7-c956-43db-b41c-ef482069e0ec None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:33.620771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3bd7-c956-43db-b41c-ef482069e0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:33.639432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aebd3bd7-c956-43db-b41c-ef482069e0ec None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:33.640770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aebd3bd7-c956-43db-b41c-ef482069e0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:33.719953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 607/1207] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:33.743599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80ad82-40f6-4164-9c4d-195b1db56f30 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:33.863684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80ad82-40f6-4164-9c4d-195b1db56f30 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:33.905549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:33.906410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:33.906735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:33.907624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:33.920142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:02:33.928841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:33.930112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:33.931051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 601/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 16:02:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:33.938160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3bd7-c956-43db-b41c-ef482069e0ec None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:33.990576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:34.057922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3bd7-c956-43db-b41c-ef482069e0ec None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:34.098915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:34.099672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:34.099977 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:34.100323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:34.101162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:34.101472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:34.103059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:34.115555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 608/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 16:02:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:34.121088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:34.178146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:34.242842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c139bba-76f5-49ba-af6c-c4a89b8cce66 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:34.284397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:34.285206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:34.285494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:34.285788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:34.296539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:34.298120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:34.318455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:34.319581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:34.325185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:34.325185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:34.325185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:34.325185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:34.325185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 602/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:34.325515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:34.435579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:34.439561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627d4e7-2aa7-42bc-8539-b643dab22276 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:34.482483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0174662-397f-444b-848f-3711b0615303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:34.483378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0174662-397f-444b-848f-3711b0615303 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:34.483799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0174662-397f-444b-848f-3711b0615303 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:34.484176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0174662-397f-444b-848f-3711b0615303 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:34.485070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0174662-397f-444b-848f-3711b0615303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:34.485784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0174662-397f-444b-848f-3711b0615303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:34.487767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0174662-397f-444b-848f-3711b0615303 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:34.508710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 609/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:34.515204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:34.639478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361f9d8-4df8-479f-91d0-971cb24a9441 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:34.641185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:34.682384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:34.683365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:34.683764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:34.684114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:34.692363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:34.694121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:02:34.710012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:34.711341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:34.712589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 603/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:02:34.720325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0174662-397f-444b-848f-3711b0615303 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:34.838591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0174662-397f-444b-848f-3711b0615303 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:34.876511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:34.877190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:34.877478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:34.877799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:34.878620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:34.878936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:34.880367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:34.901780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 610/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 16:02:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:34.908161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:35.035564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21605497-4f61-4e2d-9a43-510a81389087 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:35.078314 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:35.079160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:35.079570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:35.079862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:35.125184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:35.181043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:35.182895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:35.303584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 604/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:02:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 16:02:35.309712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:35.333524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:35.421311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18537ab-1cc4-4a42-8a80-506fe38814e3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:35.458659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:35.459603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:35.459983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:35.460352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:35.472136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:35.473624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:35.492931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:35.494000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:35.494912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:35.494912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:35.494912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:35.494912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:35.499766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 611/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:35.501311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:35.578028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:35.630044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420c303-faab-44cd-a991-dd88912d7756 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:35.671781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:35.672437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:35.673302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:35.673620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:35.674814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:35.675165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:35.676807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:35.690556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 605/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:35.697477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:35.786002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:35.827497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfcda63-f98c-4bc5-b5f3-9b0b4acd1f29 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:35.866467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:35.867228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:35.867495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:35.867801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:35.875281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:35.877250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:02:35.892028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:35.893387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:35.894368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 612/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:02:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:35.902161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:36.031444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e72abb-3823-427c-acb2-35922ce61c15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:36.066010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:36.081595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b8e0b-34ef-422e-a441-1516e91efaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:36.082294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b8e0b-34ef-422e-a441-1516e91efaae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:36.082577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b8e0b-34ef-422e-a441-1516e91efaae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:36.082857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b8e0b-34ef-422e-a441-1516e91efaae None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:36.083662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b8e0b-34ef-422e-a441-1516e91efaae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:36.083974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b8e0b-34ef-422e-a441-1516e91efaae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:36.085411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b8e0b-34ef-422e-a441-1516e91efaae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:36.100237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 606/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:02:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:36.106504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:36.230096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3216f-c6cf-4a3c-94c5-1cca12c03918 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:36.267595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:36.275030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eb960-fd0a-4595-ae23-1ccf02ba442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:36.275686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb960-fd0a-4595-ae23-1ccf02ba442e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:36.275966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb960-fd0a-4595-ae23-1ccf02ba442e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:36.276236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eb960-fd0a-4595-ae23-1ccf02ba442e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:36.277056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6eb960-fd0a-4595-ae23-1ccf02ba442e None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=7512a42da9154c008dad057c5508a02b) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:36.277388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6eb960-fd0a-4595-ae23-1ccf02ba442e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:36.278946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6eb960-fd0a-4595-ae23-1ccf02ba442e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:36.285419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 613/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:02:36 2022] GET /identity/v3/role_assignments?user.id=7512a42da9154c008dad057c5508a02b&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:38.540755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe93d5-32c3-4b35-a8a9-4a1ec8dd21ca None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:38.541312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe93d5-32c3-4b35-a8a9-4a1ec8dd21ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:38.541312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe93d5-32c3-4b35-a8a9-4a1ec8dd21ca None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:38.542736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 607/1220] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:38.549152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a56bb2-5402-4eaf-a6a8-0ee04374dd1a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:38.549972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a56bb2-5402-4eaf-a6a8-0ee04374dd1a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:38.549972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a56bb2-5402-4eaf-a6a8-0ee04374dd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:38.570899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-67a56bb2-5402-4eaf-a6a8-0ee04374dd1a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:38.572637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-67a56bb2-5402-4eaf-a6a8-0ee04374dd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:38.647042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 614/1221] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:02:38.669866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe93d5-32c3-4b35-a8a9-4a1ec8dd21ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:38.780158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe93d5-32c3-4b35-a8a9-4a1ec8dd21ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:38.818475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:38.819192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:38.819479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:38.819759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:38.832779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:02:38.843940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:38.845839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:38.846858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 608/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 16:02:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:38.852618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a56bb2-5402-4eaf-a6a8-0ee04374dd1a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:38.972359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a56bb2-5402-4eaf-a6a8-0ee04374dd1a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:39.001493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:39.012274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e310f8f-3026-453f-b485-884626e555da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:39.012949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e310f8f-3026-453f-b485-884626e555da None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:39.013236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e310f8f-3026-453f-b485-884626e555da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:39.013515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e310f8f-3026-453f-b485-884626e555da None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:39.014298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e310f8f-3026-453f-b485-884626e555da None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:39.014630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e310f8f-3026-453f-b485-884626e555da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:39.015998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e310f8f-3026-453f-b485-884626e555da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:39.024781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 615/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 16:02:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:39.031190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:39.084190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:39.158575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49dc5-54ba-470f-99f3-402f589ed512 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:39.194008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:39.194943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:39.195255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:39.195527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:39.206845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:39.208294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:39.227291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:39.228416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:39.229371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:39.229371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:39.229371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:39.229371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:39.229371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 609/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:39.234313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e310f8f-3026-453f-b485-884626e555da None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:39.347425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e310f8f-3026-453f-b485-884626e555da None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:39.384935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:39.385710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:39.386087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:39.386367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:39.387681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:39.387681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:39.389403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:39.403853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 616/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:39.410930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:39.519943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:39.527360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edd7e7-ce7c-4848-b338-8a97054587d5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:39.565217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:39.566105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:39.566423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:39.566669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:39.576103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:39.577554 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:02:39.592895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:39.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:39.595568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 610/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:39.600498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:39.650775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:39.712326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180dc4ee-4f3e-4466-971c-800f1d978348 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:39.751181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:39.751885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:39.752172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:39.752450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:39.753311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:39.753652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:39.755390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:39.774746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 617/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 16:02:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:39.780936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:39.903100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c7b5-a366-491f-94a3-282c0f159766 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:39.951621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:39.952912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:39.953400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:39.954437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:39.969987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:39.971913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:39.995873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:39.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:39.998133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:39.998133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:39.998133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:39.998133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:40.004633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 611/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 16:02:40.005893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:40.122450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e85579-4396-412d-9d73-4dbca96da54b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:40.124921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:40.166596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:40.167208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:40.167494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:40.167781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:40.168554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:40.168866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:40.170301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:40.182901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 618/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:40.188992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:40.200154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:40.304813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81ea9-2093-40a2-97b7-915dc7c9b88e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:40.340496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:40.341291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:40.341585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:40.341906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:40.349570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:40.351868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:02:40.373673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:40.374957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:40.382047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 612/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:02:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:40.382954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:40.492206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a640150-3c9c-4b97-814b-a033d109dce2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:40.530997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015f66f-0924-4b75-bece-738dc49a8095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:40.531769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015f66f-0924-4b75-bece-738dc49a8095 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:40.532093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015f66f-0924-4b75-bece-738dc49a8095 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:40.532356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015f66f-0924-4b75-bece-738dc49a8095 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:40.533254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015f66f-0924-4b75-bece-738dc49a8095 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:40.533586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015f66f-0924-4b75-bece-738dc49a8095 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:40.535290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015f66f-0924-4b75-bece-738dc49a8095 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:40.549055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 619/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:02:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:40.555166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:40.630787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:40.668367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2883adba-cb46-48d0-8b5f-93949145d0c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:40.704356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410bcfa-4e6b-4d86-8c6c-52bb8d824017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:40.705271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410bcfa-4e6b-4d86-8c6c-52bb8d824017 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:40.705549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410bcfa-4e6b-4d86-8c6c-52bb8d824017 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:40.705869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410bcfa-4e6b-4d86-8c6c-52bb8d824017 None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/7512a42da9154c008dad057c5508a02b/roles/763f046b87da4abbaff504f77c55288c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:40.721909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410bcfa-4e6b-4d86-8c6c-52bb8d824017 None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=7512a42da9154c008dad057c5508a02b, role_id=763f046b87da4abbaff504f77c55288c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:40.723342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410bcfa-4e6b-4d86-8c6c-52bb8d824017 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:40.737510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 613/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:02:40 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/7512a42da9154c008dad057c5508a02b/roles/763f046b87da4abbaff504f77c55288c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:02:43.109276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22903b28-f452-4c83-bee1-b19f5acd5af3 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:43.109887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22903b28-f452-4c83-bee1-b19f5acd5af3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:43.109981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22903b28-f452-4c83-bee1-b19f5acd5af3 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:43.111502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 620/1233] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:43.117750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe0164-7799-42dc-a885-60c6a7d51561 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:43.118189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe0164-7799-42dc-a885-60c6a7d51561 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:43.118327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe0164-7799-42dc-a885-60c6a7d51561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:43.136209 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-93fe0164-7799-42dc-a885-60c6a7d51561 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:43.137759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93fe0164-7799-42dc-a885-60c6a7d51561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:43.216117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 614/1234] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:43.240938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22903b28-f452-4c83-bee1-b19f5acd5af3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:43.274598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:43.360861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22903b28-f452-4c83-bee1-b19f5acd5af3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:43.397042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:43.397885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:43.398181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:43.398451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:43.410268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:02:43.418476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:43.419465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:43.420542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 621/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Thu Jun 16 16:02:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:43.426501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe0164-7799-42dc-a885-60c6a7d51561 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:43.479266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:43.545118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe0164-7799-42dc-a885-60c6a7d51561 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:43.581208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:43.581983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:43.582277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:43.582598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:43.583415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:43.583715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:43.585192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:43.595292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 615/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Thu Jun 16 16:02:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:43.601750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:43.696602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:43.714516 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151d3d6-cd77-46d9-b702-35eff658586d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:43.750387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:43.751386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:43.751701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:43.752112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:43.765080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:43.767464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:43.786821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:43.788014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:43.789087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:43.789087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:43.789087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:43.789087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:43.789087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 622/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:43.793838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:43.915740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc533d4f-c8df-4c2b-b759-8fab793fe50f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:43.929315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:43.963420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:43.964131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:43.964459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:43.964865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:43.965618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:43.966091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:43.967585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:43.982201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 616/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:43.988664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:44.107954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f75efb-4017-43af-a3d3-db147966a1fa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:44.151445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:44.152255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:44.152691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:44.152971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:44.156427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:44.164501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:44.166276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:02:44.182458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:44.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:44.190098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 623/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:44.190730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:44.311354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5e337-ea98-44c3-b11e-2143770b57d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:44.352690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:44.353501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:44.353844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:44.354128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:44.355087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:44.355765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:44.357225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:44.377365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 617/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 16:02:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:44.384415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:44.510662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e1634-4c67-4f5c-9e8d-e33b8af8cbb5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:44.551345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:44.552042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:44.552332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:44.552611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:44.620612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:44.628287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:44.629913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:44.742180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 624/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:02:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:02:44.748215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:44.778163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:44.860035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faec730-2084-4661-b50e-5892cf4183dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:44.899383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:44.900162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:44.900478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:44.900793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:44.913842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:44.915435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:44.940345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:44.941433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:44.942432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:44.942432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:44.942432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:44.942432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:44.942432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 618/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:44.949267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:45.056771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:45.069089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e1288-aa80-4db3-9ba8-3d785b00f457 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:45.111002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:45.111692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:45.112025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:45.112254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:45.112976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:45.113289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:45.114683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:45.126963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 625/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:45.133296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:45.225931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:45.250501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d4743-91d7-4989-b49e-91b707ff9ba6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:45.287362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:45.288240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:45.288594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:45.288829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:45.296035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:45.297995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:02:45.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:45.313674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:45.315204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 619/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:45.320199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:45.431674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a5d0e-fc48-4803-be49-7d3b417645b1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:45.471277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6b061-d542-4191-a9af-7d0fb080bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:45.472061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b061-d542-4191-a9af-7d0fb080bd89 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:45.472401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b061-d542-4191-a9af-7d0fb080bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:45.472683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b061-d542-4191-a9af-7d0fb080bd89 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:45.473677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6b061-d542-4191-a9af-7d0fb080bd89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:45.474103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6b061-d542-4191-a9af-7d0fb080bd89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:45.476653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6b061-d542-4191-a9af-7d0fb080bd89 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:45.495961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 626/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:45.497253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:45.608577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc038d2a-e36a-4cdd-8e6f-b44627498ed1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:45.645795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b021fa-94e6-434b-bddf-7b7441592afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:45.646460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b021fa-94e6-434b-bddf-7b7441592afb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:45.646746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b021fa-94e6-434b-bddf-7b7441592afb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:45.647024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b021fa-94e6-434b-bddf-7b7441592afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:45.647858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b021fa-94e6-434b-bddf-7b7441592afb None admin] RBAC: Adding query filter params (role.id=763f046b87da4abbaff504f77c55288c, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=7512a42da9154c008dad057c5508a02b) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:45.648195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b021fa-94e6-434b-bddf-7b7441592afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:45.649841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b021fa-94e6-434b-bddf-7b7441592afb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:45.655867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 620/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:02:45 2022] GET /identity/v3/role_assignments?user.id=7512a42da9154c008dad057c5508a02b&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=763f046b87da4abbaff504f77c55288c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:47.875479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33071b-6065-4b6a-97ce-aaef0d66e654 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:47.876037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33071b-6065-4b6a-97ce-aaef0d66e654 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:47.876159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33071b-6065-4b6a-97ce-aaef0d66e654 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:47.877638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 627/1247] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:02:47.883668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b903530-05f7-41fd-bf13-a6a6680cc3dc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:47.885090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b903530-05f7-41fd-bf13-a6a6680cc3dc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:47.885090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b903530-05f7-41fd-bf13-a6a6680cc3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:47.889338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:47.905210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1b903530-05f7-41fd-bf13-a6a6680cc3dc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:47.907323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1b903530-05f7-41fd-bf13-a6a6680cc3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:47.999431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 621/1248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:48.026048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33071b-6065-4b6a-97ce-aaef0d66e654 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:48.049305 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:48.156314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33071b-6065-4b6a-97ce-aaef0d66e654 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:48.198193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:48.199048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:48.199322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:48.199629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:48.212882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:02:48.221208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:48.222321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:48.223477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 628/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:48.228503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b903530-05f7-41fd-bf13-a6a6680cc3dc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:48.362740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b903530-05f7-41fd-bf13-a6a6680cc3dc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:48.401189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:48.407333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:48.408045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:48.408296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:48.408572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:48.409542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:48.409891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:48.411267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:48.420003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 622/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:02:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:48.426910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:48.521090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:48.546378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d1e-0368-41be-a664-9bb7f4d8fed8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:48.591431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:48.592403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:48.592763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:48.593199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:48.606424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:48.608049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:48.628326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:48.629443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:48.630452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:48.630452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:48.630452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:48.630452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:48.630452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 629/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:48.636315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:48.763769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292124d-f6aa-4492-bec5-008913ddff10 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:48.803519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:48.804220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:48.804515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:48.804796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:48.805570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:48.805952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:48.807579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:48.826288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 623/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:02:48.828033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:48.955987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955ac2c-1919-45e9-9b32-afaf730ddb83 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:48.990323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:49.000125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:49.000957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:49.001240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:49.001515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:49.010505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:49.011950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:02:49.027759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:49.029270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:49.030594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 630/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:49.035271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:49.101610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:49.160696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a98fc4-4b3b-45a2-bb56-9160fbc3753c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:49.209075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:49.210175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:49.210600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:49.211022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:49.212332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:49.212912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:49.215248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:49.238317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 624/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 16:02:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:49.244433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:49.362805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c5546-fd7c-4637-8d69-17fa4f5f4e78 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:49.412530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:49.413429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:49.413877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:49.414305 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:49.484221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:49.499069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:49.501102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:49.613301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 631/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:02:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:02:49.619047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:49.736477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3912a01-388c-4a2a-ba0b-f66208645a87 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:49.752293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:49.775933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:49.776733 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:49.777015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:49.777294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:49.788310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:49.789828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:49.808146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:49.809315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:49.810286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:49.810286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:49.810286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:49.810286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:49.810286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 625/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:49.816130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:49.926812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:49.944831 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9523cc-66f9-435c-984f-b2c999e59785 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:49.988699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:49.989479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:49.989890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:49.990288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:49.991160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:49.991498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:49.992982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:50.010709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 632/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:50.017003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:50.141046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cb43c-219c-4a59-9c1d-8f7404f5b239 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:50.197902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:50.198827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:50.199187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:50.199604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:50.210688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:50.212635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:02:50.232233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:50.233331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:50.234580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:50.240954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 626/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:50.241902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:50.365117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666205-52be-4c23-b8f1-641bdf0717ba None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:50.408576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414371f-56f2-44d7-a9cb-7a57ddd26003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:50.409421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414371f-56f2-44d7-a9cb-7a57ddd26003 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:50.409835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414371f-56f2-44d7-a9cb-7a57ddd26003 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:50.410366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414371f-56f2-44d7-a9cb-7a57ddd26003 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:50.411449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414371f-56f2-44d7-a9cb-7a57ddd26003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:50.411878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414371f-56f2-44d7-a9cb-7a57ddd26003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:50.413525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414371f-56f2-44d7-a9cb-7a57ddd26003 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:50.429414 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 633/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:50.437623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:50.479102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:50.588513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a84ded-970e-458e-8249-271ebc88b10f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:50.625694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b34560f-5faf-48bd-bce6-79bcc43e99cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:50.626400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34560f-5faf-48bd-bce6-79bcc43e99cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:50.626768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34560f-5faf-48bd-bce6-79bcc43e99cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:50.626965 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34560f-5faf-48bd-bce6-79bcc43e99cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:50.627809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34560f-5faf-48bd-bce6-79bcc43e99cc None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=7512a42da9154c008dad057c5508a02b) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:50.628126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34560f-5faf-48bd-bce6-79bcc43e99cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:50.629591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b34560f-5faf-48bd-bce6-79bcc43e99cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:50.645844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 627/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:02:50 2022] GET /identity/v3/role_assignments?user.id=7512a42da9154c008dad057c5508a02b&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:52.806672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff79c-48cc-4c2d-b2a1-40ee4a48a541 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:52.807247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff79c-48cc-4c2d-b2a1-40ee4a48a541 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:52.807247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ff79c-48cc-4c2d-b2a1-40ee4a48a541 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:52.808605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 634/1261] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:52.815211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeb05c-a4bd-45ee-9c7d-6e48d3985cd0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:52.815840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeb05c-a4bd-45ee-9c7d-6e48d3985cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:52.815840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeb05c-a4bd-45ee-9c7d-6e48d3985cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:52.832382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-93eeb05c-a4bd-45ee-9c7d-6e48d3985cd0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:52.833585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93eeb05c-a4bd-45ee-9c7d-6e48d3985cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:52.903098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 628/1262] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:02:52.930975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff79c-48cc-4c2d-b2a1-40ee4a48a541 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:52.944896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:53.045367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ff79c-48cc-4c2d-b2a1-40ee4a48a541 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:53.080169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:53.080869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:53.081151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:53.081426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:53.091311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:02:53.099356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:53.100544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:53.101574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 635/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:53.106765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eeb05c-a4bd-45ee-9c7d-6e48d3985cd0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:53.124755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:53.207591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eeb05c-a4bd-45ee-9c7d-6e48d3985cd0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:53.246479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:53.247156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:53.247440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:53.247715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:53.248459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:53.248767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:53.250177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:53.258766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 629/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:53.266293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:53.336371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:53.371826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188c3d-09ca-45a2-8311-57b9ea07e683 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:53.404272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:53.405064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:53.405350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:53.405636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:53.416646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:53.418103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:53.435518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:53.436656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:53.437607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:53.437607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:53.437607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:53.437607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:53.437607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 636/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:53.442617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:53.528424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:53.548768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af215b1-d29e-4d1f-8028-0a509ebf6afd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:53.582533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:53.583207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:53.583492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:53.583768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:53.584494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:53.584804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:53.586246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:53.599502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 630/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:53.605324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:53.718763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62172f-181a-4dd0-bbbe-a8c2ca94ee69 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:53.742567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:53.760301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:53.761543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:53.762104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:53.762454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:53.771760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:53.773197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:02:53.794295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:53.795512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:53.796625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:53.797162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 637/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:53.803003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:53.901695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888a5db-8d73-4231-8784-f3e753af1835 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:53.948720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:53.949466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:53.949861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:53.950110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:53.950949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:53.951262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:53.952707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:53.962213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:53.972145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 631/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 16:02:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:53.977809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:54.082352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b932a04-e0b9-4c91-b3ae-a0daf88af1ad None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:54.117634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:54.118448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:54.118735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:54.119008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:54.129313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:54.130838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:54.148183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:54.149386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:54.150755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:54.150755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:54.150755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:54.150755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:54.150755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 638/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:54.155443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:54.260109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc9abd-2430-4829-bfff-0cba936be6de None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:54.295312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:54.295976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:54.296256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:54.296531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:54.297259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:54.297592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:54.299016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:54.310799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 632/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:54.316426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:54.348711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:54.417100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3753051-7915-43c3-9823-a6265d19b5b1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:54.451075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:54.451873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:54.452160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:54.452436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:54.459552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:54.460989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:02:54.475284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:54.476834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:54.477959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 639/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:54.482782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:54.540783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:54.583163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c9ccf-6579-40e7-abf3-b8a8006a038b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:54.617894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c644c37-3f3c-4728-86bc-c57cb50ad337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:54.618569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c644c37-3f3c-4728-86bc-c57cb50ad337 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:54.618854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c644c37-3f3c-4728-86bc-c57cb50ad337 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:54.619131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c644c37-3f3c-4728-86bc-c57cb50ad337 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:54.619912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c644c37-3f3c-4728-86bc-c57cb50ad337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:54.620222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c644c37-3f3c-4728-86bc-c57cb50ad337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:54.621651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c644c37-3f3c-4728-86bc-c57cb50ad337 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:54.634650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 633/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:54.640869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:54.741407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:54.751135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0512bb1-e19f-4666-a8b5-a09fcc77bbde None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:54.788195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e163af-6037-444d-90b1-91aa238cf01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:54.789089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e163af-6037-444d-90b1-91aa238cf01a None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:54.789372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e163af-6037-444d-90b1-91aa238cf01a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:54.789649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e163af-6037-444d-90b1-91aa238cf01a None admin] PATH_INFO: `/v3/projects/af8309ed84b44895b09b00c0bb184778/users/7512a42da9154c008dad057c5508a02b/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:54.791604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e163af-6037-444d-90b1-91aa238cf01a None admin] RBAC: Authorizing `identity:create_grant(project_id=af8309ed84b44895b09b00c0bb184778, user_id=7512a42da9154c008dad057c5508a02b, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:54.793037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e163af-6037-444d-90b1-91aa238cf01a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:54.806196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 640/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Thu Jun 16 16:02:54 2022] PUT /identity/v3/projects/af8309ed84b44895b09b00c0bb184778/users/7512a42da9154c008dad057c5508a02b/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:02:56.943225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a31cc-d858-484b-89e3-1da4762a4075 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:56.943780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a31cc-d858-484b-89e3-1da4762a4075 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:56.943780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a31cc-d858-484b-89e3-1da4762a4075 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:56.945151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 634/1274] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:02:56.952100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c084-f144-4188-9441-c88321027aaf None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:56.952734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c084-f144-4188-9441-c88321027aaf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:56.952734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11c084-f144-4188-9441-c88321027aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:56.973113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8a11c084-f144-4188-9441-c88321027aaf None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:02:56.974381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8a11c084-f144-4188-9441-c88321027aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:02:57.042138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 641/1275] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:02:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:02:57.067007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a31cc-d858-484b-89e3-1da4762a4075 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:57.174223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a31cc-d858-484b-89e3-1da4762a4075 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:57.191923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:57.209802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:57.210532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:57.210817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:57.211096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:57.221426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 16:02:57.228952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:57.230116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:57.231313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 635/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Thu Jun 16 16:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:57.236710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11c084-f144-4188-9441-c88321027aaf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:57.318090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:57.337546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11c084-f144-4188-9441-c88321027aaf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:57.373553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:57.374256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:57.374543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:57.374819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:57.375635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:57.375940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:57.377829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:57.386569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 642/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Thu Jun 16 16:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:57.392467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:57.494055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb4ba-0c23-49eb-b6fe-9864ada0437b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:57.528092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:57.528885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:57.529172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:57.529455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:57.540825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:57.542324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:57.560266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:57.561454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:57.562607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:57.562607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:57.562607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:57.562607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:57.562607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 636/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:57.567220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:57.671527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01e341-5405-46bf-aef7-d2d5c04c94de None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:57.704263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:57.704955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:57.705241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:57.705520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:57.706265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:57.706573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:57.707931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:57.720217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 643/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:57.725986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:57.757111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:57.828596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15281a0-d20f-4a88-93d5-349b5f05efbb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:57.862191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:57.862996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:57.863280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:57.863560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:57.870985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:57.872441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 16:02:57.887097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:57.888224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:02:57.889204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 637/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:57.895608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:57.901249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:58.018967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ea4cd-ba63-4b8f-ab76-a0be11ad736c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:58.052496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:58.053157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:58.053443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:58.053747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:58.054580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:58.054891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:58.056325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:58.073969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 644/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Thu Jun 16 16:02:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:58.079952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:58.176499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:58.187356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc19f17-12bb-46ca-9619-5f59b7b64a7d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:58.224282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:58.224930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:58.225214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:58.225491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:58.293633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:58.295034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:58.383820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:58.394695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 638/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Thu Jun 16 16:02:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 16:02:58.400503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:58.462827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:58.503101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ca482-a04a-4148-b0c0-69dabc46b02b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:58.537732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:58.538610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:58.538894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:58.539173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:58.549194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:58.550598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 16:02:58.568051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:58.569142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:58.570507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 16:02:58.570507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 16:02:58.570507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 16:02:58.570507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:58.570507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 645/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:58.574909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:58.674368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1a286-0864-44ae-b7b8-82f6bb981656 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:58.711193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:58.711842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:58.712124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:58.712401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:58.713107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:58.713513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:58.714955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:58.730657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 639/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Thu Jun 16 16:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:58.736862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:58.836811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b817b9a-33c2-4aff-82c2-3a6c114bd9ae None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:58.851394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:58.871319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:58.872124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:58.872414 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:58.872698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:58.879419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:58.880863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 16:02:58.895294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:02:58.896458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Jun 16 16:02:58.898047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 646/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Thu Jun 16 16:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:02:58.902471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:58.952964 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:02:59.003986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb2ed1-51c3-42cd-8a6a-d218fa6f17b3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:59.038285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d45919-1b43-4aea-ba85-37609e12318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:59.038935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d45919-1b43-4aea-ba85-37609e12318b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:59.039214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d45919-1b43-4aea-ba85-37609e12318b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:59.039487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d45919-1b43-4aea-ba85-37609e12318b None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:59.040257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d45919-1b43-4aea-ba85-37609e12318b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:59.040562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d45919-1b43-4aea-ba85-37609e12318b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:59.042001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d45919-1b43-4aea-ba85-37609e12318b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:59.054422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 640/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Thu Jun 16 16:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:02:59.060453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:02:59.164579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02187ff5-3f64-4342-b83f-999e3546a1c1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:02:59.198177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da19575-5ece-4a86-90c0-150fe9a7f342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:02:59.198829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19575-5ece-4a86-90c0-150fe9a7f342 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:02:59.199114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19575-5ece-4a86-90c0-150fe9a7f342 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:02:59.199392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da19575-5ece-4a86-90c0-150fe9a7f342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:02:59.200248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da19575-5ece-4a86-90c0-150fe9a7f342 None admin] RBAC: Adding query filter params (role.id=d3e29c31885f4a7da9a2d5bf83cd3d79, scope.project.id=af8309ed84b44895b09b00c0bb184778, user.id=7512a42da9154c008dad057c5508a02b) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:02:59.200562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da19575-5ece-4a86-90c0-150fe9a7f342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:02:59.202008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da19575-5ece-4a86-90c0-150fe9a7f342 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:02:59.207708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 647/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Thu Jun 16 16:02:59 2022] GET /identity/v3/role_assignments?user.id=7512a42da9154c008dad057c5508a02b&scope.project.id=af8309ed84b44895b09b00c0bb184778&role.id=d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:03:01.343887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06cde-31e9-4338-ae27-5f447b14afdc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:01.344934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06cde-31e9-4338-ae27-5f447b14afdc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:01.345043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba06cde-31e9-4338-ae27-5f447b14afdc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:01.346356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 641/1288] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:01.351533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef998c-cd7a-4055-bc1a-7bc7d4d7e937 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:01.351953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef998c-cd7a-4055-bc1a-7bc7d4d7e937 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:01.352094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef998c-cd7a-4055-bc1a-7bc7d4d7e937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:01.369143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ceef998c-cd7a-4055-bc1a-7bc7d4d7e937 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:01.370357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ceef998c-cd7a-4055-bc1a-7bc7d4d7e937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:01.398741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:01.436044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 648/1289] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:03:01.458919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06cde-31e9-4338-ae27-5f447b14afdc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:01.560702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba06cde-31e9-4338-ae27-5f447b14afdc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:01.571196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:01.595793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:01.596597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:01.596885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:01.597170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:01.597855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:01.599232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:03:01.611389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:03:01.612633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:01.613863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 642/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:03:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:03:01.618538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef998c-cd7a-4055-bc1a-7bc7d4d7e937 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:01.715944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef998c-cd7a-4055-bc1a-7bc7d4d7e937 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:01.749824 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0a19c-8633-4c49-bc55-e4e14e571476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:01.751061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0a19c-8633-4c49-bc55-e4e14e571476 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:01.751306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0a19c-8633-4c49-bc55-e4e14e571476 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:01.751570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0a19c-8633-4c49-bc55-e4e14e571476 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:01.752265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0a19c-8633-4c49-bc55-e4e14e571476 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:03:01.752571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0a19c-8633-4c49-bc55-e4e14e571476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:01.753960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0a19c-8633-4c49-bc55-e4e14e571476 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:01.758699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 649/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 16:03:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:03:01.764406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:01.863243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1563d-bb92-41a7-ab33-52930728a92a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:01.898975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfd6a7-02cd-468a-8ccb-8f962b075268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:01.899771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd6a7-02cd-468a-8ccb-8f962b075268 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:01.900130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd6a7-02cd-468a-8ccb-8f962b075268 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:01.900484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfd6a7-02cd-468a-8ccb-8f962b075268 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:01.901315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd6a7-02cd-468a-8ccb-8f962b075268 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:03:01.901736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd6a7-02cd-468a-8ccb-8f962b075268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:01.903318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfd6a7-02cd-468a-8ccb-8f962b075268 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:01.909025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 643/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 16:03:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:03:04.089043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403b437-abf0-4e7c-8531-51ba4f1ec4ce None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:04.089776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403b437-abf0-4e7c-8531-51ba4f1ec4ce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:04.089776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3403b437-abf0-4e7c-8531-51ba4f1ec4ce None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:04.090989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 650/1293] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:04.096387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7049ec-24c9-42c7-b292-749ef1b196ec None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:04.096778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7049ec-24c9-42c7-b292-749ef1b196ec None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:04.096946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7049ec-24c9-42c7-b292-749ef1b196ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:04.114298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2f7049ec-24c9-42c7-b292-749ef1b196ec None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:04.115658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2f7049ec-24c9-42c7-b292-749ef1b196ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:04.148319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:04.186088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 644/1294] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:03:04.209258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403b437-abf0-4e7c-8531-51ba4f1ec4ce None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:04.241088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:04.311499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3403b437-abf0-4e7c-8531-51ba4f1ec4ce None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:04.345186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09176f-ec14-46c7-8d5e-0a4230615e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:04.345788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09176f-ec14-46c7-8d5e-0a4230615e87 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:04.346073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09176f-ec14-46c7-8d5e-0a4230615e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:04.346351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09176f-ec14-46c7-8d5e-0a4230615e87 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:04.347070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09176f-ec14-46c7-8d5e-0a4230615e87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:04.348437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09176f-ec14-46c7-8d5e-0a4230615e87 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:04.361836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 651/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Thu Jun 16 16:03:04 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:03:06.487771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613734f-90b4-41c0-a0a3-a2edb8fc1dd0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:06.488275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613734f-90b4-41c0-a0a3-a2edb8fc1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:06.488275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613734f-90b4-41c0-a0a3-a2edb8fc1dd0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:06.489637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 645/1296] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:06.494721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4643b6d-9c7b-4e8b-b50d-8c30ea91168c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:06.495124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4643b6d-9c7b-4e8b-b50d-8c30ea91168c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:06.495283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4643b6d-9c7b-4e8b-b50d-8c30ea91168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:06.512296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e4643b6d-9c7b-4e8b-b50d-8c30ea91168c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:06.513486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e4643b6d-9c7b-4e8b-b50d-8c30ea91168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:06.585601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 652/1297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 16:03:06.608041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613734f-90b4-41c0-a0a3-a2edb8fc1dd0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:06.707341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613734f-90b4-41c0-a0a3-a2edb8fc1dd0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:06.740635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:06.741357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:06.741642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:06.741945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:06.742599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:06.743975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:03:06.756075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:03:06.757237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:06.758100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 646/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:03:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:03:06.763233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4643b6d-9c7b-4e8b-b50d-8c30ea91168c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:06.781796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:06.862517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4643b6d-9c7b-4e8b-b50d-8c30ea91168c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:06.896346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:06.897082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:06.897424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:06.897744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:06.898713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:03:06.899165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:06.900710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:06.906380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 653/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 16:03:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:03:06.912820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:06.944211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:07.012719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774208ce-7a21-4b5e-a386-ef78c792edf9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:07.046349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccffe55-333d-42a1-b4b7-8cdb19e64c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:07.046998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffe55-333d-42a1-b4b7-8cdb19e64c99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:07.047815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffe55-333d-42a1-b4b7-8cdb19e64c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:07.048089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffe55-333d-42a1-b4b7-8cdb19e64c99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:07.048865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffe55-333d-42a1-b4b7-8cdb19e64c99 None admin] RBAC: Adding query filter params (interface=public, service_id=90dd36803a5e47eebb775c8ca24fa09b, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:03:07.049175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffe55-333d-42a1-b4b7-8cdb19e64c99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:07.050587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffe55-333d-42a1-b4b7-8cdb19e64c99 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:07.055681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 647/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Thu Jun 16 16:03:06 2022] GET /identity/v3/endpoints?service_id=90dd36803a5e47eebb775c8ca24fa09b&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:03:07.061542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:07.146063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:07.163915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036d63b-3309-42f1-ab5e-e1f35bb080d2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:07.197479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fadea9-bd46-4df0-9c35-804455ba0fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:07.198160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadea9-bd46-4df0-9c35-804455ba0fce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:07.198483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadea9-bd46-4df0-9c35-804455ba0fce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:07.198804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fadea9-bd46-4df0-9c35-804455ba0fce None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:07.199477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fadea9-bd46-4df0-9c35-804455ba0fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:07.200828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fadea9-bd46-4df0-9c35-804455ba0fce None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:07.207480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 654/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Thu Jun 16 16:03:07 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:03:09.368308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00710d51-bdba-4641-956c-2a530c18326d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:09.368866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00710d51-bdba-4641-956c-2a530c18326d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:09.368866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00710d51-bdba-4641-956c-2a530c18326d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:09.370282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 648/1302] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:03:09.375712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd2f40-d424-4760-8709-71244fe67b91 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:09.376160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd2f40-d424-4760-8709-71244fe67b91 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:09.376265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd2f40-d424-4760-8709-71244fe67b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:09.393758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5cfd2f40-d424-4760-8709-71244fe67b91 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:03:09.394866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5cfd2f40-d424-4760-8709-71244fe67b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:03:09.460972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 655/1303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:03:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 16:03:09.485092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00710d51-bdba-4641-956c-2a530c18326d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:09.583697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:09.590861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00710d51-bdba-4641-956c-2a530c18326d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:09.624527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:09.625884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:09.626172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:09.626450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:09.627101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:09.628474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 16:03:09.640332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 16:03:09.642045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Jun 16 16:03:09.643258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 649/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Thu Jun 16 16:03:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 16:03:09.648265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd2f40-d424-4760-8709-71244fe67b91 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:09.724710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:09.748437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd2f40-d424-4760-8709-71244fe67b91 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:09.781327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42962d3-6a5c-40bc-a3bb-0a7f7dc4c70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:09.782502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42962d3-6a5c-40bc-a3bb-0a7f7dc4c70b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:09.782785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42962d3-6a5c-40bc-a3bb-0a7f7dc4c70b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:09.783093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42962d3-6a5c-40bc-a3bb-0a7f7dc4c70b None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:09.783794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42962d3-6a5c-40bc-a3bb-0a7f7dc4c70b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:03:09.784101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42962d3-6a5c-40bc-a3bb-0a7f7dc4c70b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:09.785476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42962d3-6a5c-40bc-a3bb-0a7f7dc4c70b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:09.790624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 656/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Thu Jun 16 16:03:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:03:09.796449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:03:09.895857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb609d9-9cca-466b-b2d4-33760b917d0e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:03:09.929793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbc7cd-80d9-4a97-8f38-81b3f5b10ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:03:09.930374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc7cd-80d9-4a97-8f38-81b3f5b10ccc None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:03:09.930658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc7cd-80d9-4a97-8f38-81b3f5b10ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:03:09.930932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc7cd-80d9-4a97-8f38-81b3f5b10ccc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:03:09.931664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbc7cd-80d9-4a97-8f38-81b3f5b10ccc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:03:09.933042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbc7cd-80d9-4a97-8f38-81b3f5b10ccc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:03:09.951164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 650/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Thu Jun 16 16:03:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:03:19.788101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=30 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:03:19.946904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:13.361058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342437a-a1df-4df2-a50d-0e7ccc31ef77 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:13.361477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342437a-a1df-4df2-a50d-0e7ccc31ef77 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:13.361782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342437a-a1df-4df2-a50d-0e7ccc31ef77 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:13.363994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 657/1307] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:13.371076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e12296-6cff-49e5-aca1-6860caf64a87 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:13.372496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e12296-6cff-49e5-aca1-6860caf64a87 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:13.373285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e12296-6cff-49e5-aca1-6860caf64a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:13.494945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-69e12296-6cff-49e5-aca1-6860caf64a87 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:13.496616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-69e12296-6cff-49e5-aca1-6860caf64a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:13.649829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 651/1308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:13.700238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba1e5d-f92c-43b0-9419-06c233321de2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:13.700522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba1e5d-f92c-43b0-9419-06c233321de2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:13.701013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba1e5d-f92c-43b0-9419-06c233321de2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:13.703077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 658/1309] 10.222.0.88 () {54 vars in 964 bytes} [Thu Jun 16 16:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:13.711739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e19ab-0972-4c86-9354-2d5649345234 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:13.712172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e19ab-0972-4c86-9354-2d5649345234 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:13.712563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e19ab-0972-4c86-9354-2d5649345234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:13.795012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-864e19ab-0972-4c86-9354-2d5649345234 None None] MFA Rules not processed for user `33e61ed31e0943e8baf9a85412860245`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:13.796919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-864e19ab-0972-4c86-9354-2d5649345234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:13.820672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:13.884606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 652/1310] 10.222.0.88 () {58 vars in 1061 bytes} [Thu Jun 16 16:04:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:13.895303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba1e5d-f92c-43b0-9419-06c233321de2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:13.897222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4aba1e5d-f92c-43b0-9419-06c233321de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:14.037021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba1e5d-f92c-43b0-9419-06c233321de2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:14.128443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff3ff1-e0a4-4886-9081-3c82178f0f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:14.129702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff3ff1-e0a4-4886-9081-3c82178f0f13 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:14.135410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff3ff1-e0a4-4886-9081-3c82178f0f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:14.135410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff3ff1-e0a4-4886-9081-3c82178f0f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:14.135410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a5ff3ff1-e0a4-4886-9081-3c82178f0f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:14.199348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:14.267084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff3ff1-e0a4-4886-9081-3c82178f0f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:14.268544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff3ff1-e0a4-4886-9081-3c82178f0f13 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:14.292805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 659/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:16.784606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a5558-acf9-436b-a70d-c4bb12806cf2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:16.785147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a5558-acf9-436b-a70d-c4bb12806cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:16.785231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a5558-acf9-436b-a70d-c4bb12806cf2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:16.786746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 653/1312] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:16.791763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681799a-02ae-4151-a2a0-1ed1957b7e80 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:16.792152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681799a-02ae-4151-a2a0-1ed1957b7e80 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:16.792363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681799a-02ae-4151-a2a0-1ed1957b7e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:16.884660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0681799a-02ae-4151-a2a0-1ed1957b7e80 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:16.885872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0681799a-02ae-4151-a2a0-1ed1957b7e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:17.013645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 660/1313] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:17.053825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a5558-acf9-436b-a70d-c4bb12806cf2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:17.055438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d87a5558-acf9-436b-a70d-c4bb12806cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:17.173244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a5558-acf9-436b-a70d-c4bb12806cf2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:17.264111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:17.267622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bd452-c7fb-4471-8301-5f5fd41fde7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:17.268295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd452-c7fb-4471-8301-5f5fd41fde7b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:17.268975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd452-c7fb-4471-8301-5f5fd41fde7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:17.269616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd452-c7fb-4471-8301-5f5fd41fde7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:17.271693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-641bd452-c7fb-4471-8301-5f5fd41fde7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:17.390839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bd452-c7fb-4471-8301-5f5fd41fde7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:17.392236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bd452-c7fb-4471-8301-5f5fd41fde7b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:17.411746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 654/1314] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:19.914882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8d838-a5e9-4b09-8b02-4459dde049ee None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:19.915453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8d838-a5e9-4b09-8b02-4459dde049ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:19.915453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8d838-a5e9-4b09-8b02-4459dde049ee None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:19.916842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 661/1315] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:19.922370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56134ff-b210-48a2-89e4-ddd90704f351 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:19.922911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56134ff-b210-48a2-89e4-ddd90704f351 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:19.923011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56134ff-b210-48a2-89e4-ddd90704f351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:19.992913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d56134ff-b210-48a2-89e4-ddd90704f351 None None] MFA Rules not processed for user `9936dac6582a4f55a12ec17571df8f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:19.994161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d56134ff-b210-48a2-89e4-ddd90704f351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:20.049276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:20.094062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 655/1316] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:19 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:04:20.133165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8d838-a5e9-4b09-8b02-4459dde049ee None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:20.134745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eda8d838-a5e9-4b09-8b02-4459dde049ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:20.145377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:20.238549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8d838-a5e9-4b09-8b02-4459dde049ee None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:20.315902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0dbee-af07-4f8a-9992-1c30a1154011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:20.316569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dbee-af07-4f8a-9992-1c30a1154011 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:20.316859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dbee-af07-4f8a-9992-1c30a1154011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:20.317133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dbee-af07-4f8a-9992-1c30a1154011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:20.318830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2eb0dbee-af07-4f8a-9992-1c30a1154011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:20.381704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:20.428669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0dbee-af07-4f8a-9992-1c30a1154011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:20.430141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0dbee-af07-4f8a-9992-1c30a1154011 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:20.448429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 662/1317] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:20 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:23.117732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8335a-2aca-4992-bc7d-aeba1430d36b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:23.118299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8335a-2aca-4992-bc7d-aeba1430d36b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:23.118299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8335a-2aca-4992-bc7d-aeba1430d36b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:23.119854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 656/1318] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:23.125561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057fee9-bb12-4209-9c5c-19a9db25b9ba None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:23.126188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057fee9-bb12-4209-9c5c-19a9db25b9ba None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:23.126188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057fee9-bb12-4209-9c5c-19a9db25b9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:23.192056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e057fee9-bb12-4209-9c5c-19a9db25b9ba None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:23.193250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e057fee9-bb12-4209-9c5c-19a9db25b9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:23.292921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 663/1319] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:23.331348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8335a-2aca-4992-bc7d-aeba1430d36b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:23.332888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8fc8335a-2aca-4992-bc7d-aeba1430d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:23.435114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8335a-2aca-4992-bc7d-aeba1430d36b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:23.506592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c2b0-8eab-4b22-9463-677eaf71a3e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:23.507237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c2b0-8eab-4b22-9463-677eaf71a3e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:23.507523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c2b0-8eab-4b22-9463-677eaf71a3e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:23.507799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c2b0-8eab-4b22-9463-677eaf71a3e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:23.509495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-abe4c2b0-8eab-4b22-9463-677eaf71a3e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:23.518301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:23.619617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c2b0-8eab-4b22-9463-677eaf71a3e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:23.621051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c2b0-8eab-4b22-9463-677eaf71a3e5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:23.639155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 657/1320] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:26.406288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489044a-662d-4570-99f1-f67aa8fd4a0f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:26.406843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489044a-662d-4570-99f1-f67aa8fd4a0f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:26.406843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489044a-662d-4570-99f1-f67aa8fd4a0f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:26.408341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 664/1321] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:26.413449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c09623-2235-4022-a493-89a0038d7588 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:26.413907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c09623-2235-4022-a493-89a0038d7588 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:26.414058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c09623-2235-4022-a493-89a0038d7588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:26.480333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a0c09623-2235-4022-a493-89a0038d7588 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:26.481549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0c09623-2235-4022-a493-89a0038d7588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:26.545223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:26.588288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 658/1322] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:26.641165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854abc3d-0016-47f6-a677-f37dd2b7ca29 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:26.641454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854abc3d-0016-47f6-a677-f37dd2b7ca29 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:26.641753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854abc3d-0016-47f6-a677-f37dd2b7ca29 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:26.643158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 665/1323] 10.222.0.88 () {54 vars in 964 bytes} [Thu Jun 16 16:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:26.649516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589eea1-7f8a-4b20-b0a8-497e020be861 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:26.649843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589eea1-7f8a-4b20-b0a8-497e020be861 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:26.650287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589eea1-7f8a-4b20-b0a8-497e020be861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:26.730460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9589eea1-7f8a-4b20-b0a8-497e020be861 None None] MFA Rules not processed for user `33e61ed31e0943e8baf9a85412860245`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:26.731595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9589eea1-7f8a-4b20-b0a8-497e020be861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:26.819535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 659/1324] 10.222.0.88 () {58 vars in 1061 bytes} [Thu Jun 16 16:04:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:04:26.828677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854abc3d-0016-47f6-a677-f37dd2b7ca29 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:26.830282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-854abc3d-0016-47f6-a677-f37dd2b7ca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:26.849934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:26.938709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854abc3d-0016-47f6-a677-f37dd2b7ca29 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:27.014962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d689e8-7180-430d-8cfd-0b42ab9a06e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:27.015608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d689e8-7180-430d-8cfd-0b42ab9a06e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:27.015919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d689e8-7180-430d-8cfd-0b42ab9a06e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:27.016197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d689e8-7180-430d-8cfd-0b42ab9a06e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:27.017884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f9d689e8-7180-430d-8cfd-0b42ab9a06e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:27.090784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:27.133501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d689e8-7180-430d-8cfd-0b42ab9a06e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:27.137652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d689e8-7180-430d-8cfd-0b42ab9a06e3 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:27.156239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 666/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:29.576713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499185c7-09a9-4803-b4ee-2e4cf5c2ed22 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:29.577273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499185c7-09a9-4803-b4ee-2e4cf5c2ed22 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:29.577273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499185c7-09a9-4803-b4ee-2e4cf5c2ed22 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:29.578767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 660/1326] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:29.583867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b36d35-b0b7-4439-bf18-4395798384b6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:29.584296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b36d35-b0b7-4439-bf18-4395798384b6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:29.584551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b36d35-b0b7-4439-bf18-4395798384b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:29.653443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d0b36d35-b0b7-4439-bf18-4395798384b6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:29.655353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d0b36d35-b0b7-4439-bf18-4395798384b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:29.753386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 667/1327] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:29.789639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499185c7-09a9-4803-b4ee-2e4cf5c2ed22 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:29.791223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-499185c7-09a9-4803-b4ee-2e4cf5c2ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:29.855310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:29.898836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499185c7-09a9-4803-b4ee-2e4cf5c2ed22 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:29.972798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34158ea4-63d1-453e-98c9-c6eed2b0a89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:29.973293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158ea4-63d1-453e-98c9-c6eed2b0a89e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:29.973572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158ea4-63d1-453e-98c9-c6eed2b0a89e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:29.973879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158ea4-63d1-453e-98c9-c6eed2b0a89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:29.975491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-34158ea4-63d1-453e-98c9-c6eed2b0a89e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:30.081026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:30.083611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34158ea4-63d1-453e-98c9-c6eed2b0a89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:30.085515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34158ea4-63d1-453e-98c9-c6eed2b0a89e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:30.105350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 661/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:32.290377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27cd66-2b44-4db1-b31d-0357a6b8fec1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:32.290887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27cd66-2b44-4db1-b31d-0357a6b8fec1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:32.290887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27cd66-2b44-4db1-b31d-0357a6b8fec1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:32.292259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 668/1329] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:32.297802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a042b13-f036-4b3e-afd9-1619fcc98e21 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:32.298315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a042b13-f036-4b3e-afd9-1619fcc98e21 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:32.298419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a042b13-f036-4b3e-afd9-1619fcc98e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:32.366229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a042b13-f036-4b3e-afd9-1619fcc98e21 None None] MFA Rules not processed for user `9936dac6582a4f55a12ec17571df8f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:32.367428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a042b13-f036-4b3e-afd9-1619fcc98e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:32.463175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 662/1330] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:32.500535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27cd66-2b44-4db1-b31d-0357a6b8fec1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:32.502126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a27cd66-2b44-4db1-b31d-0357a6b8fec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:32.533196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:32.609304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27cd66-2b44-4db1-b31d-0357a6b8fec1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:32.682345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059cc9c8-98e3-43e8-b41c-3401e2b890d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:32.682993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cc9c8-98e3-43e8-b41c-3401e2b890d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:32.683274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cc9c8-98e3-43e8-b41c-3401e2b890d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:32.683548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059cc9c8-98e3-43e8-b41c-3401e2b890d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:32.685173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-059cc9c8-98e3-43e8-b41c-3401e2b890d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:32.765194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:32.799945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cc9c8-98e3-43e8-b41c-3401e2b890d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:32.801360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059cc9c8-98e3-43e8-b41c-3401e2b890d4 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:32.821744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 669/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:35.442714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00190f4-c7e1-4b4e-8d6d-c63c91f2b5fc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:35.443190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00190f4-c7e1-4b4e-8d6d-c63c91f2b5fc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:35.443314 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00190f4-c7e1-4b4e-8d6d-c63c91f2b5fc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:35.444721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 663/1332] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:35.450321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e592ca-7b60-497e-8911-53bdb5a4baf3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:35.450784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e592ca-7b60-497e-8911-53bdb5a4baf3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:35.450891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e592ca-7b60-497e-8911-53bdb5a4baf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:35.518201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-75e592ca-7b60-497e-8911-53bdb5a4baf3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:35.519422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-75e592ca-7b60-497e-8911-53bdb5a4baf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:35.614441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 670/1333] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:35.650188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00190f4-c7e1-4b4e-8d6d-c63c91f2b5fc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:35.651724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b00190f4-c7e1-4b4e-8d6d-c63c91f2b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:35.724877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:35.760084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00190f4-c7e1-4b4e-8d6d-c63c91f2b5fc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:35.831662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fdc8a-b7e9-4dd4-a8dd-3dec44e58e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:35.832306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fdc8a-b7e9-4dd4-a8dd-3dec44e58e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:35.832589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fdc8a-b7e9-4dd4-a8dd-3dec44e58e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:35.832866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fdc8a-b7e9-4dd4-a8dd-3dec44e58e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:35.834524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-374fdc8a-b7e9-4dd4-a8dd-3dec44e58e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:35.949891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374fdc8a-b7e9-4dd4-a8dd-3dec44e58e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:35.951326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374fdc8a-b7e9-4dd4-a8dd-3dec44e58e21 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:35.961741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:35.970677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 664/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:38.114099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cf718-4645-4571-8ef9-6e1c31023766 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:38.114641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cf718-4645-4571-8ef9-6e1c31023766 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:38.114641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cf718-4645-4571-8ef9-6e1c31023766 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:38.116017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 671/1335] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:38.121110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4051-52dc-438a-87da-b10abe4df774 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:38.121513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4051-52dc-438a-87da-b10abe4df774 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:38.121682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4051-52dc-438a-87da-b10abe4df774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:38.186950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b0ea4051-52dc-438a-87da-b10abe4df774 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:38.188150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b0ea4051-52dc-438a-87da-b10abe4df774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:38.284623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 665/1336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:38.324304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cf718-4645-4571-8ef9-6e1c31023766 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:38.325876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-692cf718-4645-4571-8ef9-6e1c31023766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:38.366247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:38.429886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cf718-4645-4571-8ef9-6e1c31023766 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:38.502927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4ab8d-8df3-4b09-84bd-9986a0987aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:38.503573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ab8d-8df3-4b09-84bd-9986a0987aaf service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:38.503852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ab8d-8df3-4b09-84bd-9986a0987aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:38.504123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4ab8d-8df3-4b09-84bd-9986a0987aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:38.505757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e1b4ab8d-8df3-4b09-84bd-9986a0987aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:38.588353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:38.614434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ab8d-8df3-4b09-84bd-9986a0987aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:38.615877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4ab8d-8df3-4b09-84bd-9986a0987aaf service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:38.634265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 672/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:41.230151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82931c3e-263c-4aa6-ab45-3db602a033be None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:41.230712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82931c3e-263c-4aa6-ab45-3db602a033be None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:41.230712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82931c3e-263c-4aa6-ab45-3db602a033be None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:41.232116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 666/1338] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:41.237756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294feff-f951-4d7c-a98f-db75763c6189 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:41.238175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294feff-f951-4d7c-a98f-db75763c6189 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:41.238324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294feff-f951-4d7c-a98f-db75763c6189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:41.303039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7294feff-f951-4d7c-a98f-db75763c6189 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:41.304276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7294feff-f951-4d7c-a98f-db75763c6189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:41.401582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 673/1339] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:41.437902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82931c3e-263c-4aa6-ab45-3db602a033be None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:41.439440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-82931c3e-263c-4aa6-ab45-3db602a033be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:41.514980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:41.543691 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82931c3e-263c-4aa6-ab45-3db602a033be None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:41.617884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16c814-a3e6-465f-9186-93b0f76190a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:41.618690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c814-a3e6-465f-9186-93b0f76190a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:41.619076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c814-a3e6-465f-9186-93b0f76190a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:41.619415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16c814-a3e6-465f-9186-93b0f76190a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:41.620984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a16c814-a3e6-465f-9186-93b0f76190a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:41.732251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16c814-a3e6-465f-9186-93b0f76190a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:41.734296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16c814-a3e6-465f-9186-93b0f76190a1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:41.752517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 667/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:45.838800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed71da1-4bbe-4c81-8565-f9ed44e48087 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:45.839312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed71da1-4bbe-4c81-8565-f9ed44e48087 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:45.839312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed71da1-4bbe-4c81-8565-f9ed44e48087 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:45.840651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 674/1341] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:45.846876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12475087-0a1d-4562-9ab3-e84967b4093a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:45.847348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12475087-0a1d-4562-9ab3-e84967b4093a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:45.847467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12475087-0a1d-4562-9ab3-e84967b4093a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:45.853461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:45.921398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-12475087-0a1d-4562-9ab3-e84967b4093a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:45.922538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-12475087-0a1d-4562-9ab3-e84967b4093a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:46.031317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 668/1342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:46.070442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed71da1-4bbe-4c81-8565-f9ed44e48087 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:46.072628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aed71da1-4bbe-4c81-8565-f9ed44e48087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:46.124727 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:46.174952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed71da1-4bbe-4c81-8565-f9ed44e48087 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:46.178569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f3331-0adf-496e-95cb-a754cafd5eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:46.179224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3331-0adf-496e-95cb-a754cafd5eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:46.179510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3331-0adf-496e-95cb-a754cafd5eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:46.179789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3331-0adf-496e-95cb-a754cafd5eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:46.188892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f3331-0adf-496e-95cb-a754cafd5eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:46.190386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f3331-0adf-496e-95cb-a754cafd5eb2 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:46.200017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 675/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:48.955565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2e079-2543-4870-92a7-10dbdd7215f1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:48.956176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2e079-2543-4870-92a7-10dbdd7215f1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:48.956176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2e079-2543-4870-92a7-10dbdd7215f1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:48.957545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 669/1344] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:48.963406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38507b-5702-453c-a0a8-86e09c67bfbd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:48.965450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38507b-5702-453c-a0a8-86e09c67bfbd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:48.965450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38507b-5702-453c-a0a8-86e09c67bfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:48.995842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8f38507b-5702-453c-a0a8-86e09c67bfbd None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:48.997085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8f38507b-5702-453c-a0a8-86e09c67bfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:49.088230 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 676/1345] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:49.137146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2e079-2543-4870-92a7-10dbdd7215f1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:49.150615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2e079-2543-4870-92a7-10dbdd7215f1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:49.155022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ba26c-845b-4a60-ba9a-1d5bde73f811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:49.155768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba26c-845b-4a60-ba9a-1d5bde73f811 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:49.156120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba26c-845b-4a60-ba9a-1d5bde73f811 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:49.156463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ba26c-845b-4a60-ba9a-1d5bde73f811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:49.166644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ba26c-845b-4a60-ba9a-1d5bde73f811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:49.168289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ba26c-845b-4a60-ba9a-1d5bde73f811 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:49.179152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 670/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:53.350732 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ea41b-aa9b-4324-a36e-2a94956b5052 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:53.351491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ea41b-aa9b-4324-a36e-2a94956b5052 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:53.351491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ea41b-aa9b-4324-a36e-2a94956b5052 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:53.352664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 677/1347] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:53.358884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0221-08c1-4e81-a5c7-8feba841c866 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:53.359525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0221-08c1-4e81-a5c7-8feba841c866 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:53.359525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0221-08c1-4e81-a5c7-8feba841c866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:53.378316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c98e0221-08c1-4e81-a5c7-8feba841c866 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:53.379530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c98e0221-08c1-4e81-a5c7-8feba841c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:53.388213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 671/1348] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:04:53.427389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ea41b-aa9b-4324-a36e-2a94956b5052 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:04:53.438291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ea41b-aa9b-4324-a36e-2a94956b5052 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:04:53.441956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b32a6-f8b1-4a87-b05c-e48ff36a9eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:04:53.442602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b32a6-f8b1-4a87-b05c-e48ff36a9eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:53.442890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b32a6-f8b1-4a87-b05c-e48ff36a9eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:53.443172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b32a6-f8b1-4a87-b05c-e48ff36a9eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:53.453215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b32a6-f8b1-4a87-b05c-e48ff36a9eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:04:53.454683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b32a6-f8b1-4a87-b05c-e48ff36a9eb3 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:04:53.465434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 678/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:04:58.596005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb95d06-b6e6-4dce-ba53-85e0c6a84d8e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:58.596912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb95d06-b6e6-4dce-ba53-85e0c6a84d8e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:58.597359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb95d06-b6e6-4dce-ba53-85e0c6a84d8e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:58.599100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 672/1350] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:04:58.604401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2e38f-c675-49ed-9da3-cada91e11e8f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:04:58.605188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2e38f-c675-49ed-9da3-cada91e11e8f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:04:58.605624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2e38f-c675-49ed-9da3-cada91e11e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:04:58.706850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-27f2e38f-c675-49ed-9da3-cada91e11e8f None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:04:58.709942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-27f2e38f-c675-49ed-9da3-cada91e11e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:04:58.745935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:04:58.808648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 679/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:04:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:03.338714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed10b-1a63-46af-9fbe-21a9da30f477 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:03.339409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed10b-1a63-46af-9fbe-21a9da30f477 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:03.339409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ed10b-1a63-46af-9fbe-21a9da30f477 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:03.341089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 673/1352] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:03.346783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5c91-2aab-489f-a450-84c4873add6b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:03.347634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5c91-2aab-489f-a450-84c4873add6b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:03.348236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba5c91-2aab-489f-a450-84c4873add6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:03.371289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9bba5c91-2aab-489f-a450-84c4873add6b None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:03.373249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=72 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:03.375464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9bba5c91-2aab-489f-a450-84c4873add6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:03.386225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 680/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:05:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:04.853858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a257e-fc1c-418b-803b-217de371fd9a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:04.854416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a257e-fc1c-418b-803b-217de371fd9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:04.854416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a257e-fc1c-418b-803b-217de371fd9a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:04.857023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 674/1354] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:04.862987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f255bfb-e0a1-4b79-a959-d4dfb7852967 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:04.863435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f255bfb-e0a1-4b79-a959-d4dfb7852967 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:04.863556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f255bfb-e0a1-4b79-a959-d4dfb7852967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:04.882851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5f255bfb-e0a1-4b79-a959-d4dfb7852967 None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:04.884009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5f255bfb-e0a1-4b79-a959-d4dfb7852967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:04.892988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 681/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:05:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:08.760530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ff45c-cfaf-42f1-9c84-4876b1b53399 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:08.761232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ff45c-cfaf-42f1-9c84-4876b1b53399 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:08.761232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ff45c-cfaf-42f1-9c84-4876b1b53399 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:08.763612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 675/1356] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:08.768911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8061af9-3ac7-4852-be6f-38fc3d668c82 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:08.769390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8061af9-3ac7-4852-be6f-38fc3d668c82 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:08.769486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8061af9-3ac7-4852-be6f-38fc3d668c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:08.790357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b8061af9-3ac7-4852-be6f-38fc3d668c82 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:08.791568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b8061af9-3ac7-4852-be6f-38fc3d668c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:08.799862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 682/1357] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:08.856034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a966c54-0724-403c-b08f-cdb924afb739 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:08.856034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a966c54-0724-403c-b08f-cdb924afb739 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:08.856034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a966c54-0724-403c-b08f-cdb924afb739 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:08.858936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 676/1358] 10.222.0.88 () {54 vars in 961 bytes} [Thu Jun 16 16:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:08.866366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7423f1-284b-49a4-99ba-a1c1ab87a33e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:08.866666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7423f1-284b-49a4-99ba-a1c1ab87a33e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:08.866953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7423f1-284b-49a4-99ba-a1c1ab87a33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:08.953465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1e7423f1-284b-49a4-99ba-a1c1ab87a33e None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:08.954705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1e7423f1-284b-49a4-99ba-a1c1ab87a33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:09.055150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 683/1359] 10.222.0.88 () {58 vars in 1058 bytes} [Thu Jun 16 16:05:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:09.063908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a966c54-0724-403c-b08f-cdb924afb739 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:09.079887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a966c54-0724-403c-b08f-cdb924afb739 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:09.086465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1b687-65dd-46b8-8b2d-894cb4d13d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:09.087141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1b687-65dd-46b8-8b2d-894cb4d13d3a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:09.087669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1b687-65dd-46b8-8b2d-894cb4d13d3a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:09.088140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1b687-65dd-46b8-8b2d-894cb4d13d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:09.099174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1b687-65dd-46b8-8b2d-894cb4d13d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:09.101105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1b687-65dd-46b8-8b2d-894cb4d13d3a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:09.117924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 677/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:05:11.526410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a0c0c-33dd-4190-9de4-4c367db0eba9 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:11.526984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a0c0c-33dd-4190-9de4-4c367db0eba9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:11.526984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a0c0c-33dd-4190-9de4-4c367db0eba9 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:11.529233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 684/1361] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:11.535378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b52cc-aae4-4524-9578-0adb9ae8f8c8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:11.535917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b52cc-aae4-4524-9578-0adb9ae8f8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:11.536027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b52cc-aae4-4524-9578-0adb9ae8f8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:11.555331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3f9b52cc-aae4-4524-9578-0adb9ae8f8c8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:11.556747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f9b52cc-aae4-4524-9578-0adb9ae8f8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:11.565198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 678/1362] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:11.615479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a0c0c-33dd-4190-9de4-4c367db0eba9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:11.632332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a0c0c-33dd-4190-9de4-4c367db0eba9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:11.639115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d24bb1-b0c6-47fa-96b9-a69cbc068d1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:11.639770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24bb1-b0c6-47fa-96b9-a69cbc068d1d service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:11.640056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24bb1-b0c6-47fa-96b9-a69cbc068d1d service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:11.640450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24bb1-b0c6-47fa-96b9-a69cbc068d1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:11.649897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d24bb1-b0c6-47fa-96b9-a69cbc068d1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:11.651708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d24bb1-b0c6-47fa-96b9-a69cbc068d1d service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:11.663292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 685/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:14.145863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67141161-d3c3-46f0-b186-aef28edbcc17 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.146386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67141161-d3c3-46f0-b186-aef28edbcc17 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.146521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67141161-d3c3-46f0-b186-aef28edbcc17 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.147870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 679/1364] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:14.153582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b066828-e273-4984-9a3a-ebe76168a8d5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.154036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b066828-e273-4984-9a3a-ebe76168a8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.154198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b066828-e273-4984-9a3a-ebe76168a8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.172851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0b066828-e273-4984-9a3a-ebe76168a8d5 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:14.174118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0b066828-e273-4984-9a3a-ebe76168a8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:14.182239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 686/1365] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:14.229757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777bbc6-2bdf-4060-842e-fa160bfdbeaa None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.230053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777bbc6-2bdf-4060-842e-fa160bfdbeaa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.230332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777bbc6-2bdf-4060-842e-fa160bfdbeaa None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.232318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 680/1366] 10.222.0.88 () {54 vars in 961 bytes} [Thu Jun 16 16:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:14.237998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967dfa-737f-41c5-be77-15b99fd4971c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.238229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967dfa-737f-41c5-be77-15b99fd4971c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.238398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967dfa-737f-41c5-be77-15b99fd4971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.258320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0c967dfa-737f-41c5-be77-15b99fd4971c None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:14.259532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0c967dfa-737f-41c5-be77-15b99fd4971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:14.269689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 687/1367] 10.222.0.88 () {58 vars in 1058 bytes} [Thu Jun 16 16:05:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:14.277853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777bbc6-2bdf-4060-842e-fa160bfdbeaa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:14.294954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777bbc6-2bdf-4060-842e-fa160bfdbeaa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:14.300356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9f256-4303-43d4-b9ed-235afcb33ae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:14.301611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f256-4303-43d4-b9ed-235afcb33ae3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:14.302747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f256-4303-43d4-b9ed-235afcb33ae3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:14.302747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9f256-4303-43d4-b9ed-235afcb33ae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:14.313240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9f256-4303-43d4-b9ed-235afcb33ae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:14.314893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9f256-4303-43d4-b9ed-235afcb33ae3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:14.324551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 681/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:16.585736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cfb0f-57cf-479c-babb-8cc53da44c3b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:16.586402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cfb0f-57cf-479c-babb-8cc53da44c3b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:16.586402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1cfb0f-57cf-479c-babb-8cc53da44c3b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:16.587831 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 688/1369] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:16.593566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa209a-ac70-466f-aa2d-2b6fa344faee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:16.594002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa209a-ac70-466f-aa2d-2b6fa344faee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:16.594246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa209a-ac70-466f-aa2d-2b6fa344faee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:16.614831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2bfa209a-ac70-466f-aa2d-2b6fa344faee None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:16.616038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2bfa209a-ac70-466f-aa2d-2b6fa344faee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:16.624385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 682/1370] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:16.670315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cfb0f-57cf-479c-babb-8cc53da44c3b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:16.686149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1cfb0f-57cf-479c-babb-8cc53da44c3b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:16.692215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d405ce-db69-4791-b9ea-6f0ea115893b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:16.693182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d405ce-db69-4791-b9ea-6f0ea115893b service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:16.693438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d405ce-db69-4791-b9ea-6f0ea115893b service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:16.693744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d405ce-db69-4791-b9ea-6f0ea115893b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:16.706229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d405ce-db69-4791-b9ea-6f0ea115893b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:16.707987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d405ce-db69-4791-b9ea-6f0ea115893b service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:16.725879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 689/1371] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:17.204498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152ded2-c020-467c-a120-0222c5b8b1df None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.204792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152ded2-c020-467c-a120-0222c5b8b1df None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.205286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152ded2-c020-467c-a120-0222c5b8b1df None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.206741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 683/1372] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:17.212266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37802aea-86c0-4659-ba32-0ea8c9a14b67 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.212564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37802aea-86c0-4659-ba32-0ea8c9a14b67 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.212849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37802aea-86c0-4659-ba32-0ea8c9a14b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.231143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-37802aea-86c0-4659-ba32-0ea8c9a14b67 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:17.232749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-37802aea-86c0-4659-ba32-0ea8c9a14b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:17.242007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 690/1373] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:17.545182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf5cd-0e51-4f0c-a086-f33250680548 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.545477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf5cd-0e51-4f0c-a086-f33250680548 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.546686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf5cd-0e51-4f0c-a086-f33250680548 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.547347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 684/1374] 10.222.0.88 () {54 vars in 963 bytes} [Thu Jun 16 16:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:17.552379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce31b3-8cd6-41c8-b2ff-0e4edfe97258 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.552804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce31b3-8cd6-41c8-b2ff-0e4edfe97258 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.553193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce31b3-8cd6-41c8-b2ff-0e4edfe97258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.561050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:17.608601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fcce31b3-8cd6-41c8-b2ff-0e4edfe97258 None None] MFA Rules not processed for user `984fa8bcb66146e5ae376a16d1d1b3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:17.609999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fcce31b3-8cd6-41c8-b2ff-0e4edfe97258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:17.674946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 691/1375] 10.222.0.88 () {58 vars in 1060 bytes} [Thu Jun 16 16:05:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:17.682548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cf5cd-0e51-4f0c-a086-f33250680548 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:17.693947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cf5cd-0e51-4f0c-a086-f33250680548 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:17.697756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c33b80-3d9a-4bcb-9847-c68633606b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:17.698576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33b80-3d9a-4bcb-9847-c68633606b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:17.699110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33b80-3d9a-4bcb-9847-c68633606b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:17.699494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c33b80-3d9a-4bcb-9847-c68633606b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:17.710029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c33b80-3d9a-4bcb-9847-c68633606b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:17.711882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c33b80-3d9a-4bcb-9847-c68633606b6e service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:17.724042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 685/1376] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:18.978288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff608e4-dcb6-47a5-8a11-d4bca82c324f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:18.978975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff608e4-dcb6-47a5-8a11-d4bca82c324f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:18.978975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff608e4-dcb6-47a5-8a11-d4bca82c324f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:18.980203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 692/1377] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:18.985902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb13d-f31e-4b42-ae82-c2686914e600 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:18.986322 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb13d-f31e-4b42-ae82-c2686914e600 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:18.986465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bb13d-f31e-4b42-ae82-c2686914e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:19.007111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aa0bb13d-f31e-4b42-ae82-c2686914e600 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:19.008601 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aa0bb13d-f31e-4b42-ae82-c2686914e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:19.021778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 686/1378] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:19.069390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff608e4-dcb6-47a5-8a11-d4bca82c324f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:19.085256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff608e4-dcb6-47a5-8a11-d4bca82c324f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:19.092470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5420dc2d-068b-4179-9324-ec92f5b49f15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:19.093187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420dc2d-068b-4179-9324-ec92f5b49f15 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:19.093477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420dc2d-068b-4179-9324-ec92f5b49f15 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:19.093787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5420dc2d-068b-4179-9324-ec92f5b49f15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:19.107067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420dc2d-068b-4179-9324-ec92f5b49f15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:19.108804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5420dc2d-068b-4179-9324-ec92f5b49f15 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:19.119406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 693/1379] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:21.358450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1353cf-4c70-45ab-9609-eb3318ace838 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:21.358980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1353cf-4c70-45ab-9609-eb3318ace838 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:21.358980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1353cf-4c70-45ab-9609-eb3318ace838 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:21.360598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 687/1380] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:21.366009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea446c9-9aa9-43df-a2e1-de08eae3ab16 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:21.366891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea446c9-9aa9-43df-a2e1-de08eae3ab16 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:21.367395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea446c9-9aa9-43df-a2e1-de08eae3ab16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:21.396518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7ea446c9-9aa9-43df-a2e1-de08eae3ab16 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:21.398279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7ea446c9-9aa9-43df-a2e1-de08eae3ab16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:21.409360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 694/1381] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:21.462795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1353cf-4c70-45ab-9609-eb3318ace838 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:21.495917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1353cf-4c70-45ab-9609-eb3318ace838 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:21.504349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74e884-c8db-407a-85d5-b1a5243179eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:21.505119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e884-c8db-407a-85d5-b1a5243179eb service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:21.505483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e884-c8db-407a-85d5-b1a5243179eb service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:21.505864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74e884-c8db-407a-85d5-b1a5243179eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:21.515763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74e884-c8db-407a-85d5-b1a5243179eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:21.517451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74e884-c8db-407a-85d5-b1a5243179eb service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:21.528576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 688/1382] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:23.718185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf12a38-97f0-4bdc-bbb9-27fc42a7d152 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:23.718746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf12a38-97f0-4bdc-bbb9-27fc42a7d152 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:23.718746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf12a38-97f0-4bdc-bbb9-27fc42a7d152 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:23.720658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 695/1383] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:23.725751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cc3be-de13-4c3d-9b7c-72507e2fdbaf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:23.726199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cc3be-de13-4c3d-9b7c-72507e2fdbaf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:23.726626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cc3be-de13-4c3d-9b7c-72507e2fdbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:23.745237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ef1cc3be-de13-4c3d-9b7c-72507e2fdbaf None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:23.747189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef1cc3be-de13-4c3d-9b7c-72507e2fdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:23.755780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 689/1384] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:23.799373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf12a38-97f0-4bdc-bbb9-27fc42a7d152 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:23.814419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf12a38-97f0-4bdc-bbb9-27fc42a7d152 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:23.819983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e98e9-a33e-489f-ab6b-b89854191a4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:23.820625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e98e9-a33e-489f-ab6b-b89854191a4e service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:23.820909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e98e9-a33e-489f-ab6b-b89854191a4e service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:23.821191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e98e9-a33e-489f-ab6b-b89854191a4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:23.831483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e98e9-a33e-489f-ab6b-b89854191a4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:23.832739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e98e9-a33e-489f-ab6b-b89854191a4e service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:23.844795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 696/1385] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:26.136416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27947077-2e59-4f86-8ef0-9e3442b62354 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:26.137089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27947077-2e59-4f86-8ef0-9e3442b62354 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:26.137736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27947077-2e59-4f86-8ef0-9e3442b62354 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:26.140019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 690/1386] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:26.147025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a463b-0e97-43e4-9936-706ac5f85d36 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:26.147470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a463b-0e97-43e4-9936-706ac5f85d36 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:26.147815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a463b-0e97-43e4-9936-706ac5f85d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:26.170447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ab9a463b-0e97-43e4-9936-706ac5f85d36 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:26.172101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ab9a463b-0e97-43e4-9936-706ac5f85d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:26.193152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 697/1387] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:26.250437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27947077-2e59-4f86-8ef0-9e3442b62354 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:26.265946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27947077-2e59-4f86-8ef0-9e3442b62354 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:26.271676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38719541-fbd6-4546-816b-f2518779b87c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:26.272354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38719541-fbd6-4546-816b-f2518779b87c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:26.272655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38719541-fbd6-4546-816b-f2518779b87c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:26.273021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38719541-fbd6-4546-816b-f2518779b87c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:26.283422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38719541-fbd6-4546-816b-f2518779b87c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:26.285163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38719541-fbd6-4546-816b-f2518779b87c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:26.297800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 691/1388] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:28.593075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78787de0-767d-43d1-aa15-9c0379f4a5be None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:28.593655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78787de0-767d-43d1-aa15-9c0379f4a5be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:28.593852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78787de0-767d-43d1-aa15-9c0379f4a5be None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:28.595871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 698/1389] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:28.601488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75555c9-387a-4728-bf72-90e908dc3c24 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:28.602013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75555c9-387a-4728-bf72-90e908dc3c24 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:28.602130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75555c9-387a-4728-bf72-90e908dc3c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:28.620835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e75555c9-387a-4728-bf72-90e908dc3c24 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:28.622650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e75555c9-387a-4728-bf72-90e908dc3c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:28.630890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 692/1390] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:28.681010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78787de0-767d-43d1-aa15-9c0379f4a5be None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:28.703580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78787de0-767d-43d1-aa15-9c0379f4a5be None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:28.709871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f841728-1ac8-418e-bc84-ce47d5ef4a66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:28.710558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f841728-1ac8-418e-bc84-ce47d5ef4a66 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:28.710893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f841728-1ac8-418e-bc84-ce47d5ef4a66 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:28.711308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f841728-1ac8-418e-bc84-ce47d5ef4a66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:28.720966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f841728-1ac8-418e-bc84-ce47d5ef4a66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:28.722632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f841728-1ac8-418e-bc84-ce47d5ef4a66 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:28.733863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 699/1391] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:31.011664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730f13a-80ec-448d-a205-9c1572c89971 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:31.012738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730f13a-80ec-448d-a205-9c1572c89971 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:31.012859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730f13a-80ec-448d-a205-9c1572c89971 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:31.014371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 693/1392] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:31.021026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63244e-0b77-45a5-ab1a-e5a238ee64ad None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:31.021504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63244e-0b77-45a5-ab1a-e5a238ee64ad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:31.021808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63244e-0b77-45a5-ab1a-e5a238ee64ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:31.040519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7e63244e-0b77-45a5-ab1a-e5a238ee64ad None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:31.041745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7e63244e-0b77-45a5-ab1a-e5a238ee64ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:31.051495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 700/1393] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:31.098180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730f13a-80ec-448d-a205-9c1572c89971 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:31.119119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730f13a-80ec-448d-a205-9c1572c89971 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:31.126073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b40e8b-1040-4368-baa9-a23d0ae81b9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:31.126982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b40e8b-1040-4368-baa9-a23d0ae81b9f service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:31.127696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b40e8b-1040-4368-baa9-a23d0ae81b9f service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:31.128056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b40e8b-1040-4368-baa9-a23d0ae81b9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:31.143617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b40e8b-1040-4368-baa9-a23d0ae81b9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:31.145821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b40e8b-1040-4368-baa9-a23d0ae81b9f service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:31.160277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 694/1394] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:33.290988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113e9b-78f9-4787-81b9-ac08fdfa5c1a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.291521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113e9b-78f9-4787-81b9-ac08fdfa5c1a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.291521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75113e9b-78f9-4787-81b9-ac08fdfa5c1a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.292916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 701/1395] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:33.298692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90830238-fd80-4a49-8258-40df334edf97 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.299122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90830238-fd80-4a49-8258-40df334edf97 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.299272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90830238-fd80-4a49-8258-40df334edf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.317795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-90830238-fd80-4a49-8258-40df334edf97 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:33.319034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90830238-fd80-4a49-8258-40df334edf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:33.324795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f46281-ce6f-4c5c-ac99-139ad66a2d4d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.325090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f46281-ce6f-4c5c-ac99-139ad66a2d4d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.325369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f46281-ce6f-4c5c-ac99-139ad66a2d4d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.326773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 702/1396] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:05:33.328954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 695/1397] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:33.332262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22c953-c141-4a76-a5a0-c979b29dac50 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.332572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22c953-c141-4a76-a5a0-c979b29dac50 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.332836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22c953-c141-4a76-a5a0-c979b29dac50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.351254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2e22c953-c141-4a76-a5a0-c979b29dac50 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:33.352440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2e22c953-c141-4a76-a5a0-c979b29dac50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:33.361757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 703/1398] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:05:33.385337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07238e19-9470-495e-9292-64d4d50abb7c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.385631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07238e19-9470-495e-9292-64d4d50abb7c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.386176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07238e19-9470-495e-9292-64d4d50abb7c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.388068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 696/1399] 10.222.0.88 () {54 vars in 962 bytes} [Thu Jun 16 16:05:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:33.393695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26564859-1759-4a63-a14c-bcab7b2f246c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.393695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26564859-1759-4a63-a14c-bcab7b2f246c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.393920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26564859-1759-4a63-a14c-bcab7b2f246c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.405924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07238e19-9470-495e-9292-64d4d50abb7c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:33.414228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:33.422265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07238e19-9470-495e-9292-64d4d50abb7c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:33.427763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:33.428407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.428692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.428983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.437853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:33.439565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:33.443722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-26564859-1759-4a63-a14c-bcab7b2f246c None None] MFA Rules not processed for user `982fcdf935f64b98aae10f206a21da2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:33.445007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-26564859-1759-4a63-a14c-bcab7b2f246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:33.450139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 697/1400] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:05:33.491641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:33.509258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 704/1401] 10.222.0.88 () {58 vars in 1059 bytes} [Thu Jun 16 16:05:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:33.517811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:33.527655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d047b-d948-4d30-81d1-0677b20015dc service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:33.531678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6195-1b9e-48f0-a5fd-d1638a76546e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:33.532452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6195-1b9e-48f0-a5fd-d1638a76546e service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:33.532819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6195-1b9e-48f0-a5fd-d1638a76546e service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:33.533192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6195-1b9e-48f0-a5fd-d1638a76546e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:33.544051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e6195-1b9e-48f0-a5fd-d1638a76546e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:33.545536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e6195-1b9e-48f0-a5fd-d1638a76546e service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:33.556863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 698/1402] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:34.286606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446dabb-057c-4fd8-82b3-8fea45b45e95 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:34.286921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446dabb-057c-4fd8-82b3-8fea45b45e95 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:34.287195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446dabb-057c-4fd8-82b3-8fea45b45e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:34.335919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f446dabb-057c-4fd8-82b3-8fea45b45e95 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:34.337120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f446dabb-057c-4fd8-82b3-8fea45b45e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:34.406945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 705/1403] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:05:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:34.451181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29007bd0-1366-4ae0-b926-ee7c5e06041d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:34.451696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29007bd0-1366-4ae0-b926-ee7c5e06041d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:34.451803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29007bd0-1366-4ae0-b926-ee7c5e06041d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:34.453233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 699/1404] 10.222.0.88 () {54 vars in 942 bytes} [Thu Jun 16 16:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:34.459593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961e599-b0b6-4476-80c4-4874be152ed0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:34.459953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961e599-b0b6-4476-80c4-4874be152ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:34.460244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961e599-b0b6-4476-80c4-4874be152ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:34.514715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e961e599-b0b6-4476-80c4-4874be152ed0 None None] MFA Rules not processed for user `9b4b1f0775ba4875a550ea986ed483e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:34.515958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e961e599-b0b6-4476-80c4-4874be152ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:34.592628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 706/1405] 10.222.0.88 () {58 vars in 1039 bytes} [Thu Jun 16 16:05:34 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:05:34.603154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29007bd0-1366-4ae0-b926-ee7c5e06041d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:34.617281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29007bd0-1366-4ae0-b926-ee7c5e06041d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:34.623265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abeab4-e955-4c4e-b5d2-5b6661327d2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:34.623913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abeab4-e955-4c4e-b5d2-5b6661327d2a service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:34.624194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abeab4-e955-4c4e-b5d2-5b6661327d2a service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:34.624469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abeab4-e955-4c4e-b5d2-5b6661327d2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:34.633577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abeab4-e955-4c4e-b5d2-5b6661327d2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:34.635225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abeab4-e955-4c4e-b5d2-5b6661327d2a service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:34.644137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 700/1406] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:05:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:05:35.674455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3958d-d40a-4564-b22a-1dc0e207c711 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:35.675024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3958d-d40a-4564-b22a-1dc0e207c711 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:35.675024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3958d-d40a-4564-b22a-1dc0e207c711 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:35.676362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 707/1407] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:35.682079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30783-4501-4ce2-9932-b2ce8b4cb3ce None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:35.682536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30783-4501-4ce2-9932-b2ce8b4cb3ce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:35.682775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30783-4501-4ce2-9932-b2ce8b4cb3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:35.703835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-24a30783-4501-4ce2-9932-b2ce8b4cb3ce None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:35.705061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-24a30783-4501-4ce2-9932-b2ce8b4cb3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:35.721862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 701/1408] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:35.764976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3958d-d40a-4564-b22a-1dc0e207c711 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:05:35.779904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f3958d-d40a-4564-b22a-1dc0e207c711 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:05:35.785645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377d6e3-3691-4480-ae86-2bc7a7f2afcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:05:35.786319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377d6e3-3691-4480-ae86-2bc7a7f2afcc service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:35.786599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377d6e3-3691-4480-ae86-2bc7a7f2afcc service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:35.786873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377d6e3-3691-4480-ae86-2bc7a7f2afcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:35.796556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377d6e3-3691-4480-ae86-2bc7a7f2afcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:05:35.798326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377d6e3-3691-4480-ae86-2bc7a7f2afcc service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:05:35.809914 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 708/1409] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:05:45.695292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=20 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:45.803834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:05:57.568257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e4ae-9faa-454e-8e37-0a68b8473b72 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:57.568806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e4ae-9faa-454e-8e37-0a68b8473b72 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:57.568806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817e4ae-9faa-454e-8e37-0a68b8473b72 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:57.570245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 702/1410] 10.222.0.88 () {54 vars in 918 bytes} [Thu Jun 16 16:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:57.575577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe51c-e415-41d6-8f86-c9136459db7b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:57.576036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe51c-e415-41d6-8f86-c9136459db7b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:57.576144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efe51c-e415-41d6-8f86-c9136459db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:57.594589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e9efe51c-e415-41d6-8f86-c9136459db7b None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:57.596027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e9efe51c-e415-41d6-8f86-c9136459db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:57.604487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 709/1411] 10.222.0.88 () {58 vars in 1015 bytes} [Thu Jun 16 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:05:57.619319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318a3a9-dbba-482d-9153-676a6665f06c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:57.619609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318a3a9-dbba-482d-9153-676a6665f06c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:57.619926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318a3a9-dbba-482d-9153-676a6665f06c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:57.621344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 703/1412] 10.222.0.88 () {54 vars in 918 bytes} [Thu Jun 16 16:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:05:57.626046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b102e-214a-4023-9473-bc80ca79fb76 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:05:57.626353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b102e-214a-4023-9473-bc80ca79fb76 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:05:57.626630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b102e-214a-4023-9473-bc80ca79fb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:05:57.644348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2d1b102e-214a-4023-9473-bc80ca79fb76 None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:05:57.645503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2d1b102e-214a-4023-9473-bc80ca79fb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:05:57.655111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 710/1413] 10.222.0.88 () {58 vars in 1015 bytes} [Thu Jun 16 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:00.306472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de582955-c13a-4dba-8cd0-a2825817bb9a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:00.307020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de582955-c13a-4dba-8cd0-a2825817bb9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:00.307020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de582955-c13a-4dba-8cd0-a2825817bb9a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:00.308618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:00.317156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552214f-58b1-4e73-b713-4ae7be43f4b9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:00.317586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552214f-58b1-4e73-b713-4ae7be43f4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:00.317742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552214f-58b1-4e73-b713-4ae7be43f4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:00.340142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f552214f-58b1-4e73-b713-4ae7be43f4b9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:00.342665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f552214f-58b1-4e73-b713-4ae7be43f4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:00.351985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:00.395578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de582955-c13a-4dba-8cd0-a2825817bb9a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:00.412020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de582955-c13a-4dba-8cd0-a2825817bb9a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:00.417337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f229368-c816-4f52-afbf-80756bb5b9e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:00.418198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f229368-c816-4f52-afbf-80756bb5b9e3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:00.418458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f229368-c816-4f52-afbf-80756bb5b9e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:00.418741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f229368-c816-4f52-afbf-80756bb5b9e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:00.427923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f229368-c816-4f52-afbf-80756bb5b9e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:00.429593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f229368-c816-4f52-afbf-80756bb5b9e3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:00.439564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 705/1416] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:01.402144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd2128-f558-4e31-abcd-288fd365d554 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:01.402634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd2128-f558-4e31-abcd-288fd365d554 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:01.402634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd2128-f558-4e31-abcd-288fd365d554 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:01.404038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:01.409068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f437-588f-46a3-9fa1-1f1db0ee506f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:01.409515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f437-588f-46a3-9fa1-1f1db0ee506f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:01.409644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f437-588f-46a3-9fa1-1f1db0ee506f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:01.427900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-59f7f437-588f-46a3-9fa1-1f1db0ee506f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:01.429380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-59f7f437-588f-46a3-9fa1-1f1db0ee506f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:01.436985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:01.596413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd2128-f558-4e31-abcd-288fd365d554 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:01.611269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd2128-f558-4e31-abcd-288fd365d554 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:01.616700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f40ed-cdfb-4423-a50d-a0c69689d746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:01.617333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f40ed-cdfb-4423-a50d-a0c69689d746 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:01.617616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f40ed-cdfb-4423-a50d-a0c69689d746 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:01.617927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f40ed-cdfb-4423-a50d-a0c69689d746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:01.626362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f40ed-cdfb-4423-a50d-a0c69689d746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:01.627945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f40ed-cdfb-4423-a50d-a0c69689d746 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:01.637604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 713/1419] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:02.579629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182615e6-4676-46e6-ae66-5d459089aca9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:02.580176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182615e6-4676-46e6-ae66-5d459089aca9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:02.580176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182615e6-4676-46e6-ae66-5d459089aca9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:02.581609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:02.586928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28916217-3dc3-4d8a-b959-748ce78a701e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:02.587406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28916217-3dc3-4d8a-b959-748ce78a701e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:02.587513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28916217-3dc3-4d8a-b959-748ce78a701e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:02.609060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-28916217-3dc3-4d8a-b959-748ce78a701e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:02.610315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-28916217-3dc3-4d8a-b959-748ce78a701e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:02.618353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:02.772099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182615e6-4676-46e6-ae66-5d459089aca9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:02.787927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182615e6-4676-46e6-ae66-5d459089aca9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:02.793947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e178b-00de-4c7c-b81b-f7dd0e4723e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:02.794682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e178b-00de-4c7c-b81b-f7dd0e4723e3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:02.794989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e178b-00de-4c7c-b81b-f7dd0e4723e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:02.795267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e178b-00de-4c7c-b81b-f7dd0e4723e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:02.804515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e178b-00de-4c7c-b81b-f7dd0e4723e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:02.806318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e178b-00de-4c7c-b81b-f7dd0e4723e3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:02.816224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 708/1422] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:03.755335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284cf3-e606-4234-b501-88b53b63e354 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:03.755868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284cf3-e606-4234-b501-88b53b63e354 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:03.755992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284cf3-e606-4234-b501-88b53b63e354 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:03.757333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:03.763053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659333e6-2109-4d6c-ab90-179d89611710 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:03.763457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659333e6-2109-4d6c-ab90-179d89611710 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:03.763607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659333e6-2109-4d6c-ab90-179d89611710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:03.781123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-659333e6-2109-4d6c-ab90-179d89611710 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:03.782337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-659333e6-2109-4d6c-ab90-179d89611710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:03.790142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:06:03.941921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7284cf3-e606-4234-b501-88b53b63e354 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:03.957338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7284cf3-e606-4234-b501-88b53b63e354 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:03.963021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc70f54-3317-4ff4-b833-cd341aac51ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:03.963661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc70f54-3317-4ff4-b833-cd341aac51ce service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:03.964519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc70f54-3317-4ff4-b833-cd341aac51ce service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:03.964819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc70f54-3317-4ff4-b833-cd341aac51ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:03.973527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc70f54-3317-4ff4-b833-cd341aac51ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:03.975151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc70f54-3317-4ff4-b833-cd341aac51ce service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:03.985223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 716/1425] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:04.895551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcc29e-3b17-4d88-81bb-70c43d7d51ac None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:04.896129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcc29e-3b17-4d88-81bb-70c43d7d51ac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:04.896129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcc29e-3b17-4d88-81bb-70c43d7d51ac None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:04.897533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:04.902544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a0f32-666f-4765-b321-0409fafe5ecb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:04.902997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a0f32-666f-4765-b321-0409fafe5ecb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:04.903095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a0f32-666f-4765-b321-0409fafe5ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:04.921958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8b3a0f32-666f-4765-b321-0409fafe5ecb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:04.923176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8b3a0f32-666f-4765-b321-0409fafe5ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:04.930886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:06:05.084612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcc29e-3b17-4d88-81bb-70c43d7d51ac None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:05.100005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcc29e-3b17-4d88-81bb-70c43d7d51ac None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:05.105877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ded4d7-4e50-446b-b3c8-c27542aee7cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:05.106507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ded4d7-4e50-446b-b3c8-c27542aee7cd service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:05.106787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ded4d7-4e50-446b-b3c8-c27542aee7cd service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:05.107068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ded4d7-4e50-446b-b3c8-c27542aee7cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:05.115381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ded4d7-4e50-446b-b3c8-c27542aee7cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:05.116958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ded4d7-4e50-446b-b3c8-c27542aee7cd service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:05.126256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 711/1428] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:05.978404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b792e-f706-4ab1-b741-5b45543b65a9 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:05.978991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b792e-f706-4ab1-b741-5b45543b65a9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:05.978991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b792e-f706-4ab1-b741-5b45543b65a9 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:05.980371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:05.985427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9895de8-d76e-48df-8ffc-4b503eb81797 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:05.985881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9895de8-d76e-48df-8ffc-4b503eb81797 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:05.986025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9895de8-d76e-48df-8ffc-4b503eb81797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:06.003905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a9895de8-d76e-48df-8ffc-4b503eb81797 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:06.005093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a9895de8-d76e-48df-8ffc-4b503eb81797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:06.013884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:06.164080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b792e-f706-4ab1-b741-5b45543b65a9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:06.178925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b792e-f706-4ab1-b741-5b45543b65a9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:06.184376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858df00-bf6c-4556-98ff-5e9e491224e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:06.185042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858df00-bf6c-4556-98ff-5e9e491224e9 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:06.185325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858df00-bf6c-4556-98ff-5e9e491224e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:06.185605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858df00-bf6c-4556-98ff-5e9e491224e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:06.193792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858df00-bf6c-4556-98ff-5e9e491224e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:06.195378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858df00-bf6c-4556-98ff-5e9e491224e9 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:06.206990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 719/1431] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:07.069696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837be8c5-a558-4fe9-8164-3615ba7f3147 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:07.070294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837be8c5-a558-4fe9-8164-3615ba7f3147 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:07.070294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837be8c5-a558-4fe9-8164-3615ba7f3147 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:07.071666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:07.077151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc241d-1527-4dd4-985c-35d822b8693c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:07.077550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc241d-1527-4dd4-985c-35d822b8693c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:07.077765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc241d-1527-4dd4-985c-35d822b8693c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:07.096287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4cfc241d-1527-4dd4-985c-35d822b8693c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:07.097484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4cfc241d-1527-4dd4-985c-35d822b8693c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:07.105278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:07.257284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837be8c5-a558-4fe9-8164-3615ba7f3147 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:07.272464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837be8c5-a558-4fe9-8164-3615ba7f3147 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:07.278529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9bdcc-5e63-43d8-ad0c-a2f9e9deb459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:07.279174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9bdcc-5e63-43d8-ad0c-a2f9e9deb459 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:07.279460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9bdcc-5e63-43d8-ad0c-a2f9e9deb459 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:07.279736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9bdcc-5e63-43d8-ad0c-a2f9e9deb459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:07.288034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9bdcc-5e63-43d8-ad0c-a2f9e9deb459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:07.289618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9bdcc-5e63-43d8-ad0c-a2f9e9deb459 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:07.299082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 714/1434] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:08.153309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a8ce-52b4-4f62-9d86-067e5fd2d763 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:08.153788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a8ce-52b4-4f62-9d86-067e5fd2d763 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:08.153932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a8ce-52b4-4f62-9d86-067e5fd2d763 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:08.155384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:08.160328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9fa49-517c-4cf3-b8e6-1857edb76744 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:08.160749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9fa49-517c-4cf3-b8e6-1857edb76744 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:08.160882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9fa49-517c-4cf3-b8e6-1857edb76744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:08.178730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d3f9fa49-517c-4cf3-b8e6-1857edb76744 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:08.179907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d3f9fa49-517c-4cf3-b8e6-1857edb76744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:08.188211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:08.341759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a8ce-52b4-4f62-9d86-067e5fd2d763 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:08.357130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a8ce-52b4-4f62-9d86-067e5fd2d763 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:08.362472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923afed2-3884-46e8-8160-92e82b20e3c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:08.363106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923afed2-3884-46e8-8160-92e82b20e3c8 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:08.363390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923afed2-3884-46e8-8160-92e82b20e3c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:08.363668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923afed2-3884-46e8-8160-92e82b20e3c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:08.372507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923afed2-3884-46e8-8160-92e82b20e3c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:08.374131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923afed2-3884-46e8-8160-92e82b20e3c8 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:08.383959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 722/1437] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:09.231826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47b652-077b-49dc-8977-2f1ba929b093 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:09.232366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47b652-077b-49dc-8977-2f1ba929b093 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:09.232366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47b652-077b-49dc-8977-2f1ba929b093 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:09.233792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:09.238846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cae99-812e-429f-a482-f70a47e821e8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:09.239265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cae99-812e-429f-a482-f70a47e821e8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:09.239416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5cae99-812e-429f-a482-f70a47e821e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:09.257859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3c5cae99-812e-429f-a482-f70a47e821e8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:09.259066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3c5cae99-812e-429f-a482-f70a47e821e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:09.267113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:09.417973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47b652-077b-49dc-8977-2f1ba929b093 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:09.432110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47b652-077b-49dc-8977-2f1ba929b093 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:09.437392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bde180-bbbf-4696-93b5-ba07c2f3cbf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:09.438063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde180-bbbf-4696-93b5-ba07c2f3cbf8 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:09.438344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde180-bbbf-4696-93b5-ba07c2f3cbf8 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:09.438622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde180-bbbf-4696-93b5-ba07c2f3cbf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:09.446750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bde180-bbbf-4696-93b5-ba07c2f3cbf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:09.448857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bde180-bbbf-4696-93b5-ba07c2f3cbf8 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:09.458285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 717/1440] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:11.508374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a437e-9e24-45e6-870f-ab1fa952f8ad None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:11.508913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a437e-9e24-45e6-870f-ab1fa952f8ad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:11.508913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a437e-9e24-45e6-870f-ab1fa952f8ad None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:11.510827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:11.516089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82c02-5928-4165-9272-c05d4ff626fa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:11.516536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82c02-5928-4165-9272-c05d4ff626fa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:11.516645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82c02-5928-4165-9272-c05d4ff626fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:11.534549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3aa82c02-5928-4165-9272-c05d4ff626fa None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:11.535728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3aa82c02-5928-4165-9272-c05d4ff626fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:11.543686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:11.581951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a437e-9e24-45e6-870f-ab1fa952f8ad None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:11.593440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a437e-9e24-45e6-870f-ab1fa952f8ad None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:11.597394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7f289-fb14-423f-9af2-acf545989272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:11.598074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f289-fb14-423f-9af2-acf545989272 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:11.598361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f289-fb14-423f-9af2-acf545989272 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:11.598640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7f289-fb14-423f-9af2-acf545989272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:11.607817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7f289-fb14-423f-9af2-acf545989272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:11.609321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7f289-fb14-423f-9af2-acf545989272 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:11.619016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 725/1443] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:13.912317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b06f13-0d09-4cc7-b050-33c5e1f4db1f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:13.912891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b06f13-0d09-4cc7-b050-33c5e1f4db1f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:13.912891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b06f13-0d09-4cc7-b050-33c5e1f4db1f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:13.914341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:13.920306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031317-5ffe-49fa-93ac-1a4a728a4b84 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:13.920747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031317-5ffe-49fa-93ac-1a4a728a4b84 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:13.920840 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a031317-5ffe-49fa-93ac-1a4a728a4b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:13.938433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9a031317-5ffe-49fa-93ac-1a4a728a4b84 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:13.939619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9a031317-5ffe-49fa-93ac-1a4a728a4b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:13.948444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:13.986768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b06f13-0d09-4cc7-b050-33c5e1f4db1f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:13.997401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b06f13-0d09-4cc7-b050-33c5e1f4db1f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:14.000900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489950-5e73-4af2-981b-8511ed268a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:14.001536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489950-5e73-4af2-981b-8511ed268a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:14.001862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489950-5e73-4af2-981b-8511ed268a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:14.002141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489950-5e73-4af2-981b-8511ed268a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:14.011699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489950-5e73-4af2-981b-8511ed268a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:14.013438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489950-5e73-4af2-981b-8511ed268a98 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:14.022768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 720/1446] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:16.449277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759096e2-d894-433e-b19c-106a580770c1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:16.449853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759096e2-d894-433e-b19c-106a580770c1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:16.449853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759096e2-d894-433e-b19c-106a580770c1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:16.451450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:16.456572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ff23f-5fac-44a2-8093-63f391e8b1bc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:16.456956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ff23f-5fac-44a2-8093-63f391e8b1bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:16.457136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ff23f-5fac-44a2-8093-63f391e8b1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:16.475574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2d8ff23f-5fac-44a2-8093-63f391e8b1bc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:16.476752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d8ff23f-5fac-44a2-8093-63f391e8b1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:16.484928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:16.522294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759096e2-d894-433e-b19c-106a580770c1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:16.533838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759096e2-d894-433e-b19c-106a580770c1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:16.537293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63fcd6-7601-428c-a329-cd293568728e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:16.537967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63fcd6-7601-428c-a329-cd293568728e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:16.538247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63fcd6-7601-428c-a329-cd293568728e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:16.538523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63fcd6-7601-428c-a329-cd293568728e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:16.547921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63fcd6-7601-428c-a329-cd293568728e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:16.549344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63fcd6-7601-428c-a329-cd293568728e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:16.559289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 728/1449] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:18.812951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658d24-77a4-43b3-b5f6-368bada2f354 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:18.813538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658d24-77a4-43b3-b5f6-368bada2f354 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:18.813538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658d24-77a4-43b3-b5f6-368bada2f354 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:18.815047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:18.820338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad9647-c6cc-4648-a540-04bcc74265ba None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:18.820781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad9647-c6cc-4648-a540-04bcc74265ba None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:18.820890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad9647-c6cc-4648-a540-04bcc74265ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:18.838813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-14ad9647-c6cc-4648-a540-04bcc74265ba None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:18.840009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-14ad9647-c6cc-4648-a540-04bcc74265ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:18.847958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:18.886869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658d24-77a4-43b3-b5f6-368bada2f354 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:18.897407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658d24-77a4-43b3-b5f6-368bada2f354 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:18.901399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ae4b-d223-45ac-92cd-7d2c5a695240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:18.902066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ae4b-d223-45ac-92cd-7d2c5a695240 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:18.902351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ae4b-d223-45ac-92cd-7d2c5a695240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:18.902629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ae4b-d223-45ac-92cd-7d2c5a695240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:18.912057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7ae4b-d223-45ac-92cd-7d2c5a695240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:18.913558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7ae4b-d223-45ac-92cd-7d2c5a695240 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:18.924040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 723/1452] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:21.092580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc8020-4b1e-4a40-b70a-3151be713ef6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:21.093156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc8020-4b1e-4a40-b70a-3151be713ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:21.093156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc8020-4b1e-4a40-b70a-3151be713ef6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:21.094892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:21.100337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63bed2-fece-4b0d-81d9-89fdd73e9075 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:21.100758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63bed2-fece-4b0d-81d9-89fdd73e9075 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:21.100899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63bed2-fece-4b0d-81d9-89fdd73e9075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:21.118299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5e63bed2-fece-4b0d-81d9-89fdd73e9075 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:21.120012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e63bed2-fece-4b0d-81d9-89fdd73e9075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:21.127584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:21.165454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc8020-4b1e-4a40-b70a-3151be713ef6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:21.176451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc8020-4b1e-4a40-b70a-3151be713ef6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:21.179933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81631aed-1f2a-45b1-bad8-c8967a0fec45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:21.180562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81631aed-1f2a-45b1-bad8-c8967a0fec45 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:21.180840 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81631aed-1f2a-45b1-bad8-c8967a0fec45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:21.181143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81631aed-1f2a-45b1-bad8-c8967a0fec45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:21.190427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81631aed-1f2a-45b1-bad8-c8967a0fec45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:21.191837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81631aed-1f2a-45b1-bad8-c8967a0fec45 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:21.202293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 731/1455] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:23.377122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163805-1d30-489a-8c17-3cb578126106 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:23.377679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163805-1d30-489a-8c17-3cb578126106 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:23.377679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32163805-1d30-489a-8c17-3cb578126106 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:23.379121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:23.384421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7195a-4c41-4724-9c3f-f4cf143a6774 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:23.384887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7195a-4c41-4724-9c3f-f4cf143a6774 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:23.385524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7195a-4c41-4724-9c3f-f4cf143a6774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:23.403561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4cb7195a-4c41-4724-9c3f-f4cf143a6774 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:23.404759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4cb7195a-4c41-4724-9c3f-f4cf143a6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:23.412427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:23.450577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163805-1d30-489a-8c17-3cb578126106 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:23.461217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32163805-1d30-489a-8c17-3cb578126106 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:23.465547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7981e4a-1fe6-4285-a535-1bfc953cb4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:23.466230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7981e4a-1fe6-4285-a535-1bfc953cb4ee service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:23.466516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7981e4a-1fe6-4285-a535-1bfc953cb4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:23.466793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7981e4a-1fe6-4285-a535-1bfc953cb4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:23.475866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7981e4a-1fe6-4285-a535-1bfc953cb4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:23.477290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7981e4a-1fe6-4285-a535-1bfc953cb4ee service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:23.486813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 726/1458] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:25.649122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8441274-0dfa-401b-ba80-ff02768e7a73 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:25.649599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8441274-0dfa-401b-ba80-ff02768e7a73 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:25.649711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8441274-0dfa-401b-ba80-ff02768e7a73 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:25.651624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:25.656771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c63784-3599-4c2b-86be-a6124df1aa5f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:25.657191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c63784-3599-4c2b-86be-a6124df1aa5f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:25.657324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c63784-3599-4c2b-86be-a6124df1aa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:25.674923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-97c63784-3599-4c2b-86be-a6124df1aa5f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:25.676099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97c63784-3599-4c2b-86be-a6124df1aa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:25.683982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:25.722233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8441274-0dfa-401b-ba80-ff02768e7a73 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:25.723957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c8441274-0dfa-401b-ba80-ff02768e7a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:25.737167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8441274-0dfa-401b-ba80-ff02768e7a73 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:25.741685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523f124-215c-4086-a5f8-b4ff773cddac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:25.742359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f124-215c-4086-a5f8-b4ff773cddac service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:25.742647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f124-215c-4086-a5f8-b4ff773cddac service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:25.742928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f124-215c-4086-a5f8-b4ff773cddac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:25.752713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523f124-215c-4086-a5f8-b4ff773cddac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:25.754177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523f124-215c-4086-a5f8-b4ff773cddac service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:25.763403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 734/1461] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:27.944449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65725b-b3eb-4d30-8a8b-005962c2fd98 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:27.944992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65725b-b3eb-4d30-8a8b-005962c2fd98 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:27.944992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65725b-b3eb-4d30-8a8b-005962c2fd98 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:27.946524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:27.952373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa2f3-7b6c-4f8e-9fb0-4cd64a186544 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:27.952832 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa2f3-7b6c-4f8e-9fb0-4cd64a186544 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:27.952929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa2f3-7b6c-4f8e-9fb0-4cd64a186544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:27.959041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=79 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:06:27.972452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-60efa2f3-7b6c-4f8e-9fb0-4cd64a186544 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:27.973658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-60efa2f3-7b6c-4f8e-9fb0-4cd64a186544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:27.981636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:28.018250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65725b-b3eb-4d30-8a8b-005962c2fd98 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:28.023605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=73 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:06:28.030723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65725b-b3eb-4d30-8a8b-005962c2fd98 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:28.034209 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a314d9b-6ba2-47ff-b3e4-389550fb7bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:28.034845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a314d9b-6ba2-47ff-b3e4-389550fb7bdb service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:28.035132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a314d9b-6ba2-47ff-b3e4-389550fb7bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:28.035415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a314d9b-6ba2-47ff-b3e4-389550fb7bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:28.044383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a314d9b-6ba2-47ff-b3e4-389550fb7bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:28.045831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a314d9b-6ba2-47ff-b3e4-389550fb7bdb service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:28.055273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 729/1464] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:30.178292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36e6d3-b01c-4328-ac29-0102317f360c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:30.178810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36e6d3-b01c-4328-ac29-0102317f360c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:30.178932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36e6d3-b01c-4328-ac29-0102317f360c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:30.180236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:30.186117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dde9a-6dac-478e-891e-12776a35766a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:30.186515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dde9a-6dac-478e-891e-12776a35766a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:30.186704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218dde9a-6dac-478e-891e-12776a35766a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:30.204190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-218dde9a-6dac-478e-891e-12776a35766a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:30.205387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-218dde9a-6dac-478e-891e-12776a35766a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:30.213489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:30.250408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36e6d3-b01c-4328-ac29-0102317f360c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:30.261147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36e6d3-b01c-4328-ac29-0102317f360c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:30.264701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02620a6e-acfc-4a64-8f28-f930d8d0417d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:30.265344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620a6e-acfc-4a64-8f28-f930d8d0417d service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:30.265632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620a6e-acfc-4a64-8f28-f930d8d0417d service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:30.265942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620a6e-acfc-4a64-8f28-f930d8d0417d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:30.278115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02620a6e-acfc-4a64-8f28-f930d8d0417d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:30.279718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02620a6e-acfc-4a64-8f28-f930d8d0417d service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:30.289748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 737/1467] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:32.460398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fda23b-0380-49d6-85f0-41e67410266b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:32.460951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fda23b-0380-49d6-85f0-41e67410266b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:32.460951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fda23b-0380-49d6-85f0-41e67410266b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:32.462297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:32.468044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4770-1f82-4aea-a14e-15471258dcec None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:32.468450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4770-1f82-4aea-a14e-15471258dcec None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:32.468615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d4770-1f82-4aea-a14e-15471258dcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:32.487103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-eb0d4770-1f82-4aea-a14e-15471258dcec None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:32.488290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb0d4770-1f82-4aea-a14e-15471258dcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:32.496090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:32.532274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fda23b-0380-49d6-85f0-41e67410266b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:32.542881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fda23b-0380-49d6-85f0-41e67410266b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:32.546332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23da484-3113-4d0f-a968-6daf4e17ef97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:32.546957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23da484-3113-4d0f-a968-6daf4e17ef97 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:32.547237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23da484-3113-4d0f-a968-6daf4e17ef97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:32.547512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23da484-3113-4d0f-a968-6daf4e17ef97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:32.556508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23da484-3113-4d0f-a968-6daf4e17ef97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:32.557962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23da484-3113-4d0f-a968-6daf4e17ef97 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:32.567108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 732/1470] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:42.481972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=11 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:06:42.563105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=14 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:06:52.503865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991c24-5ca7-420d-a5aa-ff9dfb4fb567 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:52.504510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991c24-5ca7-420d-a5aa-ff9dfb4fb567 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:52.504510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991c24-5ca7-420d-a5aa-ff9dfb4fb567 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:52.505846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 739/1471] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:52.511063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c355d-cb33-4b3d-b568-558592c45c6c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:52.511477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c355d-cb33-4b3d-b568-558592c45c6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:52.511625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c355d-cb33-4b3d-b568-558592c45c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:52.529516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f8c355d-cb33-4b3d-b568-558592c45c6c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:52.530726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f8c355d-cb33-4b3d-b568-558592c45c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:52.538407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 733/1472] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:52.585430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c0d7a-1aed-4c2a-985a-2573e643dad2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:52.585636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c0d7a-1aed-4c2a-985a-2573e643dad2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:52.585942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c0d7a-1aed-4c2a-985a-2573e643dad2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:52.587842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 740/1473] 10.222.0.88 () {54 vars in 962 bytes} [Thu Jun 16 16:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:52.592769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14e211-9e95-4d73-8222-2ad1a81d1b11 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:52.593066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14e211-9e95-4d73-8222-2ad1a81d1b11 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:52.593349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14e211-9e95-4d73-8222-2ad1a81d1b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:52.611788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9a14e211-9e95-4d73-8222-2ad1a81d1b11 None None] MFA Rules not processed for user `982fcdf935f64b98aae10f206a21da2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:52.612966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a14e211-9e95-4d73-8222-2ad1a81d1b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:52.619600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 734/1474] 10.222.0.88 () {58 vars in 1059 bytes} [Thu Jun 16 16:06:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:52.626876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c0d7a-1aed-4c2a-985a-2573e643dad2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:52.637186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c0d7a-1aed-4c2a-985a-2573e643dad2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:52.640617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3edf4-b818-483d-b639-25becdf95e58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:52.641245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3edf4-b818-483d-b639-25becdf95e58 service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:52.641530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3edf4-b818-483d-b639-25becdf95e58 service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:52.641841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3edf4-b818-483d-b639-25becdf95e58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:52.651118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3edf4-b818-483d-b639-25becdf95e58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:52.652539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3edf4-b818-483d-b639-25becdf95e58 service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:52.661792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 741/1475] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:06:54.902973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34571-17e1-44dc-913d-205de81873d9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:54.903543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34571-17e1-44dc-913d-205de81873d9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:54.903543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e34571-17e1-44dc-913d-205de81873d9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:54.904902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 735/1476] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:54.909967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62420a91-de68-4af9-a8f8-b3c58342634b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:54.910436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62420a91-de68-4af9-a8f8-b3c58342634b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:54.910615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62420a91-de68-4af9-a8f8-b3c58342634b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:54.928479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-62420a91-de68-4af9-a8f8-b3c58342634b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:54.929622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-62420a91-de68-4af9-a8f8-b3c58342634b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:54.937979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 742/1477] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:54.960362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e34571-17e1-44dc-913d-205de81873d9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:54.973264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e34571-17e1-44dc-913d-205de81873d9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:54.978500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d4287-f647-4060-95d1-4ed6299203ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:54.978829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d4287-f647-4060-95d1-4ed6299203ad admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:54.979508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d4287-f647-4060-95d1-4ed6299203ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:54.979508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d4287-f647-4060-95d1-4ed6299203ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:54.980089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d4287-f647-4060-95d1-4ed6299203ad admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:54.981896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-da5d4287-f647-4060-95d1-4ed6299203ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:06:54.981896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:06:54.981896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:06:54.983044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d4287-f647-4060-95d1-4ed6299203ad admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:55.002833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 736/1478] 10.222.0.88 () {56 vars in 1108 bytes} [Thu Jun 16 16:06:54 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:06:57.158564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d646e4-3546-4183-9e49-3d980ff4d26e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:57.159147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d646e4-3546-4183-9e49-3d980ff4d26e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:57.159147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d646e4-3546-4183-9e49-3d980ff4d26e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:57.160519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 743/1479] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:57.166855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24f4d8-9361-4b3a-b3c3-84102e3aaae0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:57.167323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24f4d8-9361-4b3a-b3c3-84102e3aaae0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:57.167586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24f4d8-9361-4b3a-b3c3-84102e3aaae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:57.188484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fc24f4d8-9361-4b3a-b3c3-84102e3aaae0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:57.189707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc24f4d8-9361-4b3a-b3c3-84102e3aaae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:57.197704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 737/1480] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:57.244470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d646e4-3546-4183-9e49-3d980ff4d26e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:57.260833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d646e4-3546-4183-9e49-3d980ff4d26e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:57.266346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe561402-7c65-4b17-983d-e87d398f0500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:57.266994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe561402-7c65-4b17-983d-e87d398f0500 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:57.267277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe561402-7c65-4b17-983d-e87d398f0500 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:57.267551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe561402-7c65-4b17-983d-e87d398f0500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:57.276474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe561402-7c65-4b17-983d-e87d398f0500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:57.278111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe561402-7c65-4b17-983d-e87d398f0500 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:57.288083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 744/1481] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:06:59.656959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d269a0d-9951-4485-9c8d-71b68706b5ee None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:59.657496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d269a0d-9951-4485-9c8d-71b68706b5ee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:59.657496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d269a0d-9951-4485-9c8d-71b68706b5ee None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:59.658943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 738/1482] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:06:59.664160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce0a01-1f3d-413a-bd50-972d7759a739 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:59.664600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce0a01-1f3d-413a-bd50-972d7759a739 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:59.664752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce0a01-1f3d-413a-bd50-972d7759a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:59.682037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4fce0a01-1f3d-413a-bd50-972d7759a739 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:06:59.683147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4fce0a01-1f3d-413a-bd50-972d7759a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:06:59.691263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 745/1483] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:06:59.728560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d269a0d-9951-4485-9c8d-71b68706b5ee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:06:59.739432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d269a0d-9951-4485-9c8d-71b68706b5ee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:06:59.742960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78801cb1-90b5-451b-b908-e161aadb0a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:06:59.743607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801cb1-90b5-451b-b908-e161aadb0a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:06:59.743891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801cb1-90b5-451b-b908-e161aadb0a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:06:59.744169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801cb1-90b5-451b-b908-e161aadb0a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:06:59.752953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78801cb1-90b5-451b-b908-e161aadb0a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:06:59.754426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78801cb1-90b5-451b-b908-e161aadb0a22 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:06:59.764184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 739/1484] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:07:01.959810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff83a-1392-4e36-bda0-5d7d337cf44a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:01.960371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff83a-1392-4e36-bda0-5d7d337cf44a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:01.960371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff83a-1392-4e36-bda0-5d7d337cf44a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:01.961770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 746/1485] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:07:01.967101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee99118-fff4-44c9-9cd2-1415da7466db None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:01.967506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee99118-fff4-44c9-9cd2-1415da7466db None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:01.967703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee99118-fff4-44c9-9cd2-1415da7466db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:01.985932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aee99118-fff4-44c9-9cd2-1415da7466db None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:01.987128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aee99118-fff4-44c9-9cd2-1415da7466db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:01.995636 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 740/1486] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:02.033733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff83a-1392-4e36-bda0-5d7d337cf44a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:02.044917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff83a-1392-4e36-bda0-5d7d337cf44a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:02.048348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2d7f3-b6b0-41bd-b70d-8ee1ebe75ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:02.048980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2d7f3-b6b0-41bd-b70d-8ee1ebe75ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:02.049259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2d7f3-b6b0-41bd-b70d-8ee1ebe75ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:02.049534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2d7f3-b6b0-41bd-b70d-8ee1ebe75ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:02.058678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2d7f3-b6b0-41bd-b70d-8ee1ebe75ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:02.060144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2d7f3-b6b0-41bd-b70d-8ee1ebe75ed9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:02.069788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 747/1487] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:07:04.312284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0bfbd-7e96-4dfd-9edd-dccfb30462fc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:04.312847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0bfbd-7e96-4dfd-9edd-dccfb30462fc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:04.312847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0bfbd-7e96-4dfd-9edd-dccfb30462fc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:04.314269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 741/1488] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:07:04.319820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014888d-d1f2-409d-b7e0-af02e37c79c1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:04.320288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014888d-d1f2-409d-b7e0-af02e37c79c1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:04.320385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014888d-d1f2-409d-b7e0-af02e37c79c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:04.339002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2014888d-d1f2-409d-b7e0-af02e37c79c1 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:04.340212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2014888d-d1f2-409d-b7e0-af02e37c79c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:04.348399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 748/1489] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:04.389522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0bfbd-7e96-4dfd-9edd-dccfb30462fc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:04.400672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0bfbd-7e96-4dfd-9edd-dccfb30462fc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:04.404224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5ed9f-7022-4e19-96ed-2055e9dbe7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:04.404863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5ed9f-7022-4e19-96ed-2055e9dbe7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:04.405141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5ed9f-7022-4e19-96ed-2055e9dbe7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:04.405416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5ed9f-7022-4e19-96ed-2055e9dbe7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:04.415484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5ed9f-7022-4e19-96ed-2055e9dbe7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:04.417289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5ed9f-7022-4e19-96ed-2055e9dbe7a7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:04.427922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 742/1490] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:07:06.861596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403870d5-1d81-48a4-8a07-eea69e03e433 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:06.862319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403870d5-1d81-48a4-8a07-eea69e03e433 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:06.862319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403870d5-1d81-48a4-8a07-eea69e03e433 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:06.863589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 749/1491] 10.222.0.88 () {54 vars in 926 bytes} [Thu Jun 16 16:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:07:06.869325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ab226-59cb-4bcc-a6c5-748346ab86ce None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:06.869806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ab226-59cb-4bcc-a6c5-748346ab86ce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:06.869916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ab226-59cb-4bcc-a6c5-748346ab86ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:06.887557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ad7ab226-59cb-4bcc-a6c5-748346ab86ce None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:07:06.888762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ad7ab226-59cb-4bcc-a6c5-748346ab86ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:07:06.896372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 743/1492] 10.222.0.88 () {58 vars in 1023 bytes} [Thu Jun 16 16:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:07:06.936240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403870d5-1d81-48a4-8a07-eea69e03e433 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:07:06.949438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403870d5-1d81-48a4-8a07-eea69e03e433 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:07:06.953188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f059b6-7ff0-41b6-bdce-be9fa83dd120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:07:06.953885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f059b6-7ff0-41b6-bdce-be9fa83dd120 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:07:06.954164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f059b6-7ff0-41b6-bdce-be9fa83dd120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:07:06.954442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f059b6-7ff0-41b6-bdce-be9fa83dd120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:07:06.963519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f059b6-7ff0-41b6-bdce-be9fa83dd120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:07:06.964942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f059b6-7ff0-41b6-bdce-be9fa83dd120 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:07:06.974766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 750/1493] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:07:16.882084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:07:16.970333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=33 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:08:32.084019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20219bf5-5ccc-4d41-8eef-f7f618610627 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.084628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20219bf5-5ccc-4d41-8eef-f7f618610627 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.084628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20219bf5-5ccc-4d41-8eef-f7f618610627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.102341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-20219bf5-5ccc-4d41-8eef-f7f618610627 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:08:32.103512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-20219bf5-5ccc-4d41-8eef-f7f618610627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:08:32.111564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 744/1494] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:08:32.121401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c66ea-5bf1-4550-8426-54d9168b06fe None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.121918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c66ea-5bf1-4550-8426-54d9168b06fe None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.122021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c66ea-5bf1-4550-8426-54d9168b06fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.141423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-228c66ea-5bf1-4550-8426-54d9168b06fe None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:08:32.142667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-228c66ea-5bf1-4550-8426-54d9168b06fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:08:32.151263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 751/1495] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:08:32.155797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20219bf5-5ccc-4d41-8eef-f7f618610627 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.166020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20219bf5-5ccc-4d41-8eef-f7f618610627 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.170523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.171179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.171465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.171742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.172456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:08:32.172795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.173847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:32.173847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:08:32.173847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:08:32.174731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.186132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 745/1496] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:08:32.194044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c66ea-5bf1-4550-8426-54d9168b06fe None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.203303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c66ea-5bf1-4550-8426-54d9168b06fe None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.208404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.209051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.209335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.209612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.210321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.212131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:32.212131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: warnings.warn(msg) Jun 16 16:08:32.212131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: Jun 16 16:08:32.213330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.249638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 752/1497] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:08:32 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:08:32.254399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.264107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c1266-ee80-4e9c-b477-dd474b9cb2b6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.323646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.324321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.324611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.324895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.325580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.326647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:32.326647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:08:32.326647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:08:32.327905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.349802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 746/1498] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:08:32 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:08:32.353954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.364434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8334a-56ab-469a-9381-b1ed465d2cfd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.369033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.369691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.370011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.370291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.371007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.372012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:32.372012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: warnings.warn(msg) Jun 16 16:08:32.372012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: Jun 16 16:08:32.372893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.385890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 753/1499] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:08:32.390088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.400499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828ae4a-3da6-4f98-abd4-a729bbe4aaf3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.404942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a7ed2-3589-4f07-82b9-01920bc7addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.405598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7ed2-3589-4f07-82b9-01920bc7addb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.405914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7ed2-3589-4f07-82b9-01920bc7addb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.406194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7ed2-3589-4f07-82b9-01920bc7addb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.406915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a7ed2-3589-4f07-82b9-01920bc7addb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.407931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-881a7ed2-3589-4f07-82b9-01920bc7addb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:32.407931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:08:32.407931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:08:32.408802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a7ed2-3589-4f07-82b9-01920bc7addb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.422630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 747/1500] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:08:32.426713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.436486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51010204-6d7a-4bec-933f-5e0cff66437e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.441044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6da4d-1c75-4e65-a0db-f4c38aac6366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.442444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6da4d-1c75-4e65-a0db-f4c38aac6366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.442729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6da4d-1c75-4e65-a0db-f4c38aac6366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.443004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6da4d-1c75-4e65-a0db-f4c38aac6366 admin admin] PATH_INFO: `/v3/projects/3a351375e7af4689b1d3d604c48e4873/users/76f649f0114f410187baa12d3e5f15c9/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.457238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6da4d-1c75-4e65-a0db-f4c38aac6366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a351375e7af4689b1d3d604c48e4873, user_id=76f649f0114f410187baa12d3e5f15c9, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.458970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-bca6da4d-1c75-4e65-a0db-f4c38aac6366 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:32.458970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: warnings.warn(msg) Jun 16 16:08:32.458970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: Jun 16 16:08:32.460885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6da4d-1c75-4e65-a0db-f4c38aac6366 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.474776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 754/1501] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:08:32 2022] PUT /identity/v3/projects/3a351375e7af4689b1d3d604c48e4873/users/76f649f0114f410187baa12d3e5f15c9/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:08:32.486284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd04594-9ed4-4cee-bac9-9e307f611cb5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.486579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd04594-9ed4-4cee-bac9-9e307f611cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.486861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd04594-9ed4-4cee-bac9-9e307f611cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.513158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4dd04594-9ed4-4cee-bac9-9e307f611cb5 None None] MFA Rules not processed for user `76f649f0114f410187baa12d3e5f15c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:08:32.514397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4dd04594-9ed4-4cee-bac9-9e307f611cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:08:32.577058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 748/1502] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:08:32.590796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66647592-b334-432b-a68d-6c00f287e00a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.591094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66647592-b334-432b-a68d-6c00f287e00a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.591382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66647592-b334-432b-a68d-6c00f287e00a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.592748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 755/1503] 10.222.0.88 () {54 vars in 963 bytes} [Thu Jun 16 16:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:08:32.598118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebfcc6-6954-4dc1-9656-64647c5e437d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.598953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebfcc6-6954-4dc1-9656-64647c5e437d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.599234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebfcc6-6954-4dc1-9656-64647c5e437d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.617546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-11ebfcc6-6954-4dc1-9656-64647c5e437d None None] MFA Rules not processed for user `984fa8bcb66146e5ae376a16d1d1b3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:08:32.618715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11ebfcc6-6954-4dc1-9656-64647c5e437d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:08:32.661245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:08:32.670289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 749/1504] 10.222.0.88 () {58 vars in 1060 bytes} [Thu Jun 16 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:08:32.678610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66647592-b334-432b-a68d-6c00f287e00a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.689244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66647592-b334-432b-a68d-6c00f287e00a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.693042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.693708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.694031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.694309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.702509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.703934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.712848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 756/1505] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:08:32.757313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202738c-30f6-41e5-b2d1-a1e98f3ce153 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.757604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202738c-30f6-41e5-b2d1-a1e98f3ce153 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.757928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202738c-30f6-41e5-b2d1-a1e98f3ce153 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.759227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 750/1506] 10.222.0.88 () {52 vars in 829 bytes} [Thu Jun 16 16:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:08:32.762743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:32.824573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690401b7-ad6c-4206-b593-9d3d9621799a service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:32.829087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:32.830475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.830763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.831041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] PATH_INFO: `/v3/users/76f649f0114f410187baa12d3e5f15c9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.832227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f649f0114f410187baa12d3e5f15c9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:32.833240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:32.833240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: warnings.warn(msg) Jun 16 16:08:32.833240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: Jun 16 16:08:32.834483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:32.897626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f649f0114f410187baa12d3e5f15c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:32.898208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f649f0114f410187baa12d3e5f15c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:32.910526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f649f0114f410187baa12d3e5f15c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:32.911278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f649f0114f410187baa12d3e5f15c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:32.921688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f649f0114f410187baa12d3e5f15c9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:32.939778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:08:32.949237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 757/1507] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:08:32 2022] DELETE /identity/v3/users/76f649f0114f410187baa12d3e5f15c9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:08:32.954083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:32.954552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:32.954767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:32.973015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:08:32.974798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:08:33.033643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 751/1508] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:08:33.046137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:33.047578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:08:33.104015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec5aaa-41c0-46f7-982e-aae01e722e13 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:33.107625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a050b-5cc7-4419-b5ac-c0adf7fc395a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:33.108268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a050b-5cc7-4419-b5ac-c0adf7fc395a service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:33.108554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a050b-5cc7-4419-b5ac-c0adf7fc395a service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:33.108833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a050b-5cc7-4419-b5ac-c0adf7fc395a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:33.117743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a050b-5cc7-4419-b5ac-c0adf7fc395a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:33.119182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a050b-5cc7-4419-b5ac-c0adf7fc395a service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:33.128493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 758/1509] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:08:33.459856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:08:33.461391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:08:33.473634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ca872-b74b-4dde-8d74-e4a330b8e9f1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:08:33.478762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:08:33.479625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:08:33.479912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:08:33.480193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] PATH_INFO: `/v3/projects/3a351375e7af4689b1d3d604c48e4873` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:08:33.481406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a351375e7af4689b1d3d604c48e4873)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:08:33.482552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:08:33.482552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:08:33.482552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:08:33.483689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:08:33.546095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a351375e7af4689b1d3d604c48e4873', 'request_id': 'req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:33.556298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a351375e7af4689b1d3d604c48e4873', 'request_id': 'req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:33.570481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a351375e7af4689b1d3d604c48e4873', 'request_id': 'req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:33.581233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] The token cache is being invalidate because project 3a351375e7af4689b1d3d604c48e4873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:08:33.581522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:33.582207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bb196cf0-c74b-4e35-a60a-d4d60a4d4057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:08:33.583997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 752/1510] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:08:33 2022] DELETE /identity/v3/projects/3a351375e7af4689b1d3d604c48e4873 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:08:43.124606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:08:43.578451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:10:20.533322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640d35-e532-45d8-8cb4-cbcc497a71a0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:20.533988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640d35-e532-45d8-8cb4-cbcc497a71a0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:20.533988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e640d35-e532-45d8-8cb4-cbcc497a71a0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:20.535397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:20.541459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d9f3f-a9ca-437f-a0d4-3ff4601d0899 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:20.542016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d9f3f-a9ca-437f-a0d4-3ff4601d0899 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:20.542126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d9f3f-a9ca-437f-a0d4-3ff4601d0899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:20.561236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c9d9f3f-a9ca-437f-a0d4-3ff4601d0899 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:20.562966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c9d9f3f-a9ca-437f-a0d4-3ff4601d0899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:20.631185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:20.820466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640d35-e532-45d8-8cb4-cbcc497a71a0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:20.822212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0e640d35-e532-45d8-8cb4-cbcc497a71a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:20.897768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e640d35-e532-45d8-8cb4-cbcc497a71a0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:20.903332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf8c4d-ab62-4899-85c7-841e48435f85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:20.903989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf8c4d-ab62-4899-85c7-841e48435f85 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:20.904278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf8c4d-ab62-4899-85c7-841e48435f85 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:20.904562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf8c4d-ab62-4899-85c7-841e48435f85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:20.913516 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf8c4d-ab62-4899-85c7-841e48435f85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:20.915181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf8c4d-ab62-4899-85c7-841e48435f85 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:20.926079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 760/1513] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:21.832118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb9233-5c12-466f-bcb0-d534a4e316ca None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:21.832635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb9233-5c12-466f-bcb0-d534a4e316ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:21.832635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb9233-5c12-466f-bcb0-d534a4e316ca None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:21.834043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:21.838999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827c48-2190-452f-92e1-8de4779ca3aa None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:21.839436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827c48-2190-452f-92e1-8de4779ca3aa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:21.839571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827c48-2190-452f-92e1-8de4779ca3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:21.858519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-47827c48-2190-452f-92e1-8de4779ca3aa None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:21.860252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-47827c48-2190-452f-92e1-8de4779ca3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:21.868630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:22.024513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb9233-5c12-466f-bcb0-d534a4e316ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:22.040003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb9233-5c12-466f-bcb0-d534a4e316ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:22.045465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad7a67-b144-4b51-92bc-931a76dfc298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:22.046138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad7a67-b144-4b51-92bc-931a76dfc298 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:22.047184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad7a67-b144-4b51-92bc-931a76dfc298 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:22.047478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad7a67-b144-4b51-92bc-931a76dfc298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:22.056982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad7a67-b144-4b51-92bc-931a76dfc298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:22.058624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad7a67-b144-4b51-92bc-931a76dfc298 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:22.068527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 755/1516] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:23.213480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0fb53-4206-4e2d-8fcc-1de6dec72f0d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:23.214120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0fb53-4206-4e2d-8fcc-1de6dec72f0d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:23.214120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0fb53-4206-4e2d-8fcc-1de6dec72f0d None None] PATH_INFO: `/v3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:23.215383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 762/1517] 10.222.0.33 () {54 vars in 925 bytes} [Thu Jun 16 16:10:23 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:10:23.222737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87298a26-7e6e-4aa9-8f04-49fc038df3c1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:23.223164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87298a26-7e6e-4aa9-8f04-49fc038df3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:23.223299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87298a26-7e6e-4aa9-8f04-49fc038df3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:23.241176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-87298a26-7e6e-4aa9-8f04-49fc038df3c1 None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:23.242477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-87298a26-7e6e-4aa9-8f04-49fc038df3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:23.303429 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 756/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 16 16:10:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:31.853033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=37 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:10:32.977610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f72b8-e71b-4e1b-8c61-e96b5c46006d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:32.978244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f72b8-e71b-4e1b-8c61-e96b5c46006d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:32.978244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f72b8-e71b-4e1b-8c61-e96b5c46006d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:32.980285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 763/1519] 10.222.0.88 () {54 vars in 960 bytes} [Thu Jun 16 16:10:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:32.986219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7897d40-f40d-4c03-8e58-f5c2e76c0ac1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:32.986680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7897d40-f40d-4c03-8e58-f5c2e76c0ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:32.986796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7897d40-f40d-4c03-8e58-f5c2e76c0ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:33.006123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e7897d40-f40d-4c03-8e58-f5c2e76c0ac1 None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:33.007328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e7897d40-f40d-4c03-8e58-f5c2e76c0ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:33.015925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 757/1520] 10.222.0.88 () {58 vars in 1057 bytes} [Thu Jun 16 16:10:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:33.023467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f72b8-e71b-4e1b-8c61-e96b5c46006d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:33.038121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f72b8-e71b-4e1b-8c61-e96b5c46006d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:33.043194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcf4ed-7224-4796-8bf6-6d79b8d04247 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:33.043839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcf4ed-7224-4796-8bf6-6d79b8d04247 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:33.044133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcf4ed-7224-4796-8bf6-6d79b8d04247 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:33.044415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcf4ed-7224-4796-8bf6-6d79b8d04247 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:33.053428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcf4ed-7224-4796-8bf6-6d79b8d04247 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:33.055075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcf4ed-7224-4796-8bf6-6d79b8d04247 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:33.066247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 764/1521] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:33.403549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443888c9-30bb-47b0-9bf5-264f2e476e72 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:33.403846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443888c9-30bb-47b0-9bf5-264f2e476e72 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:33.404163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443888c9-30bb-47b0-9bf5-264f2e476e72 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:33.406060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:10:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:33.411180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5187-da61-4adf-be48-edd38f9c9f88 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:33.411476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5187-da61-4adf-be48-edd38f9c9f88 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:33.411817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a5187-da61-4adf-be48-edd38f9c9f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:33.432212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b23a5187-da61-4adf-be48-edd38f9c9f88 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:33.433413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b23a5187-da61-4adf-be48-edd38f9c9f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:33.441692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:10:33.597530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443888c9-30bb-47b0-9bf5-264f2e476e72 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:33.616940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443888c9-30bb-47b0-9bf5-264f2e476e72 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:33.622934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0dc6ab-de29-463d-b894-eba8a82a2fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:33.623670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dc6ab-de29-463d-b894-eba8a82a2fca service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:33.624169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dc6ab-de29-463d-b894-eba8a82a2fca service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:33.624502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dc6ab-de29-463d-b894-eba8a82a2fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:33.633256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dc6ab-de29-463d-b894-eba8a82a2fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:33.634960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0dc6ab-de29-463d-b894-eba8a82a2fca service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:33.645117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 759/1524] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:34.516964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b367a2-af6e-4d31-a5f6-0e0c98a0adaf None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:34.517577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b367a2-af6e-4d31-a5f6-0e0c98a0adaf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:34.517577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b367a2-af6e-4d31-a5f6-0e0c98a0adaf None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:34.518931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 16:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:34.524076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a55405-bc05-4244-aa5d-8703dc4beb81 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:34.524519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a55405-bc05-4244-aa5d-8703dc4beb81 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:34.524648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a55405-bc05-4244-aa5d-8703dc4beb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:34.543132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c2a55405-bc05-4244-aa5d-8703dc4beb81 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:34.544332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c2a55405-bc05-4244-aa5d-8703dc4beb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:34.552222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:34.705440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b367a2-af6e-4d31-a5f6-0e0c98a0adaf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:34.720868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b367a2-af6e-4d31-a5f6-0e0c98a0adaf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:34.729140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a77a8-9254-4633-afc7-f325e98d7595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:34.729675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a77a8-9254-4633-afc7-f325e98d7595 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:34.729987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a77a8-9254-4633-afc7-f325e98d7595 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:34.730265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a77a8-9254-4633-afc7-f325e98d7595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:34.739417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a77a8-9254-4633-afc7-f325e98d7595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:34.740991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a77a8-9254-4633-afc7-f325e98d7595 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:34.751154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 767/1527] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:38.887088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0407783-b398-47c6-b960-20ba5a9c129c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:38.887710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0407783-b398-47c6-b960-20ba5a9c129c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:38.887710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0407783-b398-47c6-b960-20ba5a9c129c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:38.889127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:38.894588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add99ac-5f20-489a-8d71-d955f4856d47 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:38.895033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add99ac-5f20-489a-8d71-d955f4856d47 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:38.895144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add99ac-5f20-489a-8d71-d955f4856d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:38.913291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3add99ac-5f20-489a-8d71-d955f4856d47 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:38.914531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3add99ac-5f20-489a-8d71-d955f4856d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:38.923281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:38.961712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0407783-b398-47c6-b960-20ba5a9c129c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:38.963482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b0407783-b398-47c6-b960-20ba5a9c129c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:39.024254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0407783-b398-47c6-b960-20ba5a9c129c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:39.030848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fd393-3b79-42ac-b2a4-8cabf642d8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:39.031494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fd393-3b79-42ac-b2a4-8cabf642d8ba service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:39.031777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fd393-3b79-42ac-b2a4-8cabf642d8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:39.032055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fd393-3b79-42ac-b2a4-8cabf642d8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:39.041253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fd393-3b79-42ac-b2a4-8cabf642d8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:39.042868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fd393-3b79-42ac-b2a4-8cabf642d8ba service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:39.048301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:10:39.053478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 762/1530] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:41.186070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b24223-bfc6-4a8b-9ff0-4a4c97d794e3 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:41.186639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b24223-bfc6-4a8b-9ff0-4a4c97d794e3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:41.186639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b24223-bfc6-4a8b-9ff0-4a4c97d794e3 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:41.188034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:41.193305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669659-57a3-4677-86ab-984288f6be34 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:41.193761 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669659-57a3-4677-86ab-984288f6be34 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:41.193925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48669659-57a3-4677-86ab-984288f6be34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:41.213746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-48669659-57a3-4677-86ab-984288f6be34 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:41.215129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-48669659-57a3-4677-86ab-984288f6be34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:41.224005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:10:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:41.263547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b24223-bfc6-4a8b-9ff0-4a4c97d794e3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:41.275064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b24223-bfc6-4a8b-9ff0-4a4c97d794e3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:41.279410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5509fe2-b884-4911-9ebf-0a6c16b82618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:41.280680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5509fe2-b884-4911-9ebf-0a6c16b82618 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:41.280970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5509fe2-b884-4911-9ebf-0a6c16b82618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:41.281251 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5509fe2-b884-4911-9ebf-0a6c16b82618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:41.290942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5509fe2-b884-4911-9ebf-0a6c16b82618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:41.292372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5509fe2-b884-4911-9ebf-0a6c16b82618 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:41.302656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 770/1533] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:45.026701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5110c35-c588-4d03-ae65-f53cc99abd2c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:45.027275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5110c35-c588-4d03-ae65-f53cc99abd2c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:45.027275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5110c35-c588-4d03-ae65-f53cc99abd2c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:45.028682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:45.034850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b6095-f47b-4d41-a97d-97d1fbc3aca0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:45.035384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b6095-f47b-4d41-a97d-97d1fbc3aca0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:45.035384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b6095-f47b-4d41-a97d-97d1fbc3aca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:45.054129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d22b6095-f47b-4d41-a97d-97d1fbc3aca0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:45.055363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d22b6095-f47b-4d41-a97d-97d1fbc3aca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:45.064020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:45.338463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5110c35-c588-4d03-ae65-f53cc99abd2c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:45.340162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e5110c35-c588-4d03-ae65-f53cc99abd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:45.403898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5110c35-c588-4d03-ae65-f53cc99abd2c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:45.408014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bb870-b8a6-44e9-b748-2f1e1426f9f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:45.408672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bb870-b8a6-44e9-b748-2f1e1426f9f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:45.408959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bb870-b8a6-44e9-b748-2f1e1426f9f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:45.409259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bb870-b8a6-44e9-b748-2f1e1426f9f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:45.419321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bb870-b8a6-44e9-b748-2f1e1426f9f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:45.420764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bb870-b8a6-44e9-b748-2f1e1426f9f7 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:45.430339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 765/1536] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:47.650830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d48de-e98f-45a6-ab97-c5f7e2abcb7b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:47.651435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d48de-e98f-45a6-ab97-c5f7e2abcb7b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:47.651435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d48de-e98f-45a6-ab97-c5f7e2abcb7b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:47.652754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 16:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:10:47.658440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce0c91-c3d3-49ef-a0ac-0b35424a624d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:47.658896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce0c91-c3d3-49ef-a0ac-0b35424a624d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:47.659050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce0c91-c3d3-49ef-a0ac-0b35424a624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:47.676939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a0ce0c91-c3d3-49ef-a0ac-0b35424a624d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:10:47.678167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0ce0c91-c3d3-49ef-a0ac-0b35424a624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:47.685863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 16:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:10:47.965221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d48de-e98f-45a6-ab97-c5f7e2abcb7b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:10:47.966987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b64d48de-e98f-45a6-ab97-c5f7e2abcb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:10:47.981974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d48de-e98f-45a6-ab97-c5f7e2abcb7b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:10:47.985584 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b33d94-478f-4150-b1bc-60d1e4ea625a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:10:47.986262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b33d94-478f-4150-b1bc-60d1e4ea625a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:10:47.986547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b33d94-478f-4150-b1bc-60d1e4ea625a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:10:47.986823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b33d94-478f-4150-b1bc-60d1e4ea625a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:10:47.996153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b33d94-478f-4150-b1bc-60d1e4ea625a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:10:47.997572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b33d94-478f-4150-b1bc-60d1e4ea625a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:10:48.007595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 773/1539] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:10:57.672192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:10:58.002780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=33 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:19.475567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5fa2d-ec0e-4167-84ad-7206a4d1d2ce None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.483527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5fa2d-ec0e-4167-84ad-7206a4d1d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.483527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5fa2d-ec0e-4167-84ad-7206a4d1d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.509825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-49c5fa2d-ec0e-4167-84ad-7206a4d1d2ce None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.511580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-49c5fa2d-ec0e-4167-84ad-7206a4d1d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.524718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 767/1540] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:19.535042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e0d83-3b34-411e-8a8e-b9bf0c446a30 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.535681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e0d83-3b34-411e-8a8e-b9bf0c446a30 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.535681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e0d83-3b34-411e-8a8e-b9bf0c446a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.565970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-123e0d83-3b34-411e-8a8e-b9bf0c446a30 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.567505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-123e0d83-3b34-411e-8a8e-b9bf0c446a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.583342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 774/1541] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:19.589187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5fa2d-ec0e-4167-84ad-7206a4d1d2ce None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:19.613154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5fa2d-ec0e-4167-84ad-7206a4d1d2ce None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:19.622677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:19.624398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.625019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.625537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.626685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:19.627354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:19.629261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:19.650550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 768/1542] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:19.659301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e0d83-3b34-411e-8a8e-b9bf0c446a30 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:19.678447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e0d83-3b34-411e-8a8e-b9bf0c446a30 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:19.684086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a86f6-8ea3-4fb1-b78e-d08eaa6fb723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:19.685113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a86f6-8ea3-4fb1-b78e-d08eaa6fb723 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.685482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a86f6-8ea3-4fb1-b78e-d08eaa6fb723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.685813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a86f6-8ea3-4fb1-b78e-d08eaa6fb723 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.686837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a86f6-8ea3-4fb1-b78e-d08eaa6fb723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:19.688785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a86f6-8ea3-4fb1-b78e-d08eaa6fb723 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:19.738151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 775/1543] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/projects => generated 357 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:19.741987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:19.757443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84471c46-89e1-4875-817b-2b6f6a0c3931 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.757830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84471c46-89e1-4875-817b-2b6f6a0c3931 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.758080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84471c46-89e1-4875-817b-2b6f6a0c3931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.758947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152e83-bc03-41a5-a940-a549d3491d21 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:19.792037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-84471c46-89e1-4875-817b-2b6f6a0c3931 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.793299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-84471c46-89e1-4875-817b-2b6f6a0c3931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.873698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fd16e-61af-4e32-b8e0-9f12304d255b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:19.874880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fd16e-61af-4e32-b8e0-9f12304d255b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.875440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fd16e-61af-4e32-b8e0-9f12304d255b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.876116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fd16e-61af-4e32-b8e0-9f12304d255b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.877218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fd16e-61af-4e32-b8e0-9f12304d255b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:19.879569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fd16e-61af-4e32-b8e0-9f12304d255b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:19.904420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 776/1544] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:19.906674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235c99-f04d-404c-be26-cec2e7a837f4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.907092 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235c99-f04d-404c-be26-cec2e7a837f4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.907512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235c99-f04d-404c-be26-cec2e7a837f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.923251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 769/1545] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/users => generated 279 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:19.925043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28a56-2530-4fa1-b9e4-e65a5d935d29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.925570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28a56-2530-4fa1-b9e4-e65a5d935d29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.926136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28a56-2530-4fa1-b9e4-e65a5d935d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.946901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b2235c99-f04d-404c-be26-cec2e7a837f4 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.948131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b2235c99-f04d-404c-be26-cec2e7a837f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.967733 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0ab28a56-2530-4fa1-b9e4-e65a5d935d29 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:19.969033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ab28a56-2530-4fa1-b9e4-e65a5d935d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:19.971294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 777/1546] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:19.973238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21528e0-7d00-40d4-b2a1-509825abe7a6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:19.973530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21528e0-7d00-40d4-b2a1-509825abe7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:19.973826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21528e0-7d00-40d4-b2a1-509825abe7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:19.997576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 770/1547] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:19.997576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28a56-2530-4fa1-b9e4-e65a5d935d29 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.005931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d21528e0-7d00-40d4-b2a1-509825abe7a6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.007568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d21528e0-7d00-40d4-b2a1-509825abe7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.008871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28a56-2530-4fa1-b9e4-e65a5d935d29 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.014779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3601512-5154-4ed1-b73b-308b7695500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.015455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3601512-5154-4ed1-b73b-308b7695500d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.015741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3601512-5154-4ed1-b73b-308b7695500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.016109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3601512-5154-4ed1-b73b-308b7695500d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.016970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3601512-5154-4ed1-b73b-308b7695500d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.021938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 778/1548] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.022148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3601512-5154-4ed1-b73b-308b7695500d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.022245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a951de-d434-480b-9047-04d0a65b1e71 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.022245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a951de-d434-480b-9047-04d0a65b1e71 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.022245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a951de-d434-480b-9047-04d0a65b1e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.032074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 771/1549] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:20.033385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68a5ef-b435-46e4-9d87-3dba434d0a9a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.033678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68a5ef-b435-46e4-9d87-3dba434d0a9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.034007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68a5ef-b435-46e4-9d87-3dba434d0a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.042465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-75a951de-d434-480b-9047-04d0a65b1e71 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.044052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-75a951de-d434-480b-9047-04d0a65b1e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.055860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1f68a5ef-b435-46e4-9d87-3dba434d0a9a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.057051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1f68a5ef-b435-46e4-9d87-3dba434d0a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.060655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 779/1550] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.062515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c81c-6544-420b-9b9d-c5866318b676 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.062515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c81c-6544-420b-9b9d-c5866318b676 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.062515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c81c-6544-420b-9b9d-c5866318b676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.068029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 772/1551] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.069982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcef5a0-ec83-446e-8aa0-10f3969f07cc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.069982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcef5a0-ec83-446e-8aa0-10f3969f07cc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.070672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcef5a0-ec83-446e-8aa0-10f3969f07cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.082534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9ac9c81c-6544-420b-9b9d-c5866318b676 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.083756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9ac9c81c-6544-420b-9b9d-c5866318b676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.089652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7dcef5a0-ec83-446e-8aa0-10f3969f07cc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.091385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7dcef5a0-ec83-446e-8aa0-10f3969f07cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.092549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 780/1552] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.094045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f4b5-e07b-4723-9541-f6f4cda3d003 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.094489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f4b5-e07b-4723-9541-f6f4cda3d003 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.094958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f4b5-e07b-4723-9541-f6f4cda3d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.102408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 773/1553] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.103702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcef5a0-ec83-446e-8aa0-10f3969f07cc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.114505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcef5a0-ec83-446e-8aa0-10f3969f07cc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.117771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fb50f4b5-e07b-4723-9541-f6f4cda3d003 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.118965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fb50f4b5-e07b-4723-9541-f6f4cda3d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.119094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a82be0-87ba-4c1a-9188-10e809d83bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.119684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a82be0-87ba-4c1a-9188-10e809d83bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.119976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a82be0-87ba-4c1a-9188-10e809d83bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.120250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a82be0-87ba-4c1a-9188-10e809d83bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.120975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a82be0-87ba-4c1a-9188-10e809d83bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.121282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a82be0-87ba-4c1a-9188-10e809d83bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.123134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a82be0-87ba-4c1a-9188-10e809d83bc9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.127978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 781/1554] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.128799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50f4b5-e07b-4723-9541-f6f4cda3d003 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.137986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 774/1555] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.139298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2db84-c9f0-48fd-9fa4-51584e3e2ef9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.139612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2db84-c9f0-48fd-9fa4-51584e3e2ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.139902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2db84-c9f0-48fd-9fa4-51584e3e2ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.147868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50f4b5-e07b-4723-9541-f6f4cda3d003 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.152631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f26830-7356-44bf-acbe-29e78035bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.153826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f26830-7356-44bf-acbe-29e78035bd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.154112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f26830-7356-44bf-acbe-29e78035bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.154394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f26830-7356-44bf-acbe-29e78035bd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.155119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f26830-7356-44bf-acbe-29e78035bd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.156751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f26830-7356-44bf-acbe-29e78035bd5b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.161230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d7e2db84-c9f0-48fd-9fa4-51584e3e2ef9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.162927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7e2db84-c9f0-48fd-9fa4-51584e3e2ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.169959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 782/1556] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:20.171358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f39d88-69a8-4d69-9847-0a9a19fcd803 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.171662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f39d88-69a8-4d69-9847-0a9a19fcd803 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.171949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f39d88-69a8-4d69-9847-0a9a19fcd803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.172597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 775/1557] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.173488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2db84-c9f0-48fd-9fa4-51584e3e2ef9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.183622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2db84-c9f0-48fd-9fa4-51584e3e2ef9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.189075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.189778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.190065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.190340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.191072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.191378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.192191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-22f39d88-69a8-4d69-9847-0a9a19fcd803 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.192990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.193424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-22f39d88-69a8-4d69-9847-0a9a19fcd803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.201467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 783/1558] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.202726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868e47-442b-4ab3-b437-37b75fd8e44c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.203014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868e47-442b-4ab3-b437-37b75fd8e44c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.203298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60868e47-442b-4ab3-b437-37b75fd8e44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.208883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 776/1559] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.209878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.223316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0f027-7124-4936-9f95-e86e69336475 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.223530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-60868e47-442b-4ab3-b437-37b75fd8e44c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.224607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-60868e47-442b-4ab3-b437-37b75fd8e44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.227844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.228529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.228812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.229088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.229845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.230359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.233012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.233457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 784/1560] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.234711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761c6b-f3bb-4d0a-a6e6-e55264393094 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.235024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761c6b-f3bb-4d0a-a6e6-e55264393094 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.235314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc761c6b-f3bb-4d0a-a6e6-e55264393094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.253692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 777/1561] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.253692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.254981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bc761c6b-f3bb-4d0a-a6e6-e55264393094 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.256206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bc761c6b-f3bb-4d0a-a6e6-e55264393094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.258673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58e49b-63d4-41ff-a2ab-babfac92b5c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.263226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f7a9b-271b-4054-ad7b-341a4ea5fcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.263891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f7a9b-271b-4054-ad7b-341a4ea5fcf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.264182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f7a9b-271b-4054-ad7b-341a4ea5fcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.264279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 785/1562] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.264996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761c6b-f3bb-4d0a-a6e6-e55264393094 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.266085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f7a9b-271b-4054-ad7b-341a4ea5fcf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.266889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f7a9b-271b-4054-ad7b-341a4ea5fcf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.268014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-2c9f7a9b-271b-4054-ad7b-341a4ea5fcf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:12:20.268014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:12:20.268014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:12:20.269556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f7a9b-271b-4054-ad7b-341a4ea5fcf0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.276361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc761c6b-f3bb-4d0a-a6e6-e55264393094 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.281846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.282740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.283020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.283298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] PATH_INFO: `/v3/projects/2dd88144811c46e3889b66c90df5f38f/users/b2757df3207a404a834babbf189fdf95/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.301516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 778/1563] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.301791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd88144811c46e3889b66c90df5f38f, user_id=b2757df3207a404a834babbf189fdf95, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.301903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30acc05-9b8e-41e6-ae35-3920690ba592 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.301903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30acc05-9b8e-41e6-ae35-3920690ba592 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.301903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30acc05-9b8e-41e6-ae35-3920690ba592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.302481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.321022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 786/1564] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:20 2022] PUT /identity/v3/projects/2dd88144811c46e3889b66c90df5f38f/users/b2757df3207a404a834babbf189fdf95/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:20.322120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.329422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e30acc05-9b8e-41e6-ae35-3920690ba592 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.331451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e30acc05-9b8e-41e6-ae35-3920690ba592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.361820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:20.377135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:20.396215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb243e-6474-4438-8637-afaca8a396de admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.401194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.401912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.402202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.402483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.403282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.403617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.404121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 779/1565] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.404684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:12:20.404684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: warnings.warn(msg) Jun 16 16:12:20.404684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: Jun 16 16:12:20.405020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30acc05-9b8e-41e6-ae35-3920690ba592 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.405582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.414677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30acc05-9b8e-41e6-ae35-3920690ba592 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.421427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.421602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 787/1566] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.422194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.422337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.422438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.422714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.423565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.425559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.434885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2372f6e-75c7-469f-8370-49b7935c4d3b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.439485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.440147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.440430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.440706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.441803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.441803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.443382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.456686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 780/1567] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.456686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.464135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 788/1568] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.464135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.476811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed71bb-d629-4549-a7ef-6ee9c57fdb7b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.550244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc70dc-ea6a-4023-b5ce-1eb4c8ab99f8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.555154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6193f0d-9f6d-485d-84b1-6c2d73baf16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.555853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193f0d-9f6d-485d-84b1-6c2d73baf16b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.556138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193f0d-9f6d-485d-84b1-6c2d73baf16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.556416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193f0d-9f6d-485d-84b1-6c2d73baf16b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.557117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6193f0d-9f6d-485d-84b1-6c2d73baf16b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.559089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6193f0d-9f6d-485d-84b1-6c2d73baf16b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.560023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.560716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.561013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.561283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.562049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.562359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.563968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.575878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 789/1569] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.576674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.587000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc46ca1-410a-4a71-b7b5-ff3feeb7b33a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.592769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.593448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.593749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.594037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.594733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.595784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 781/1570] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 393 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.596332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:12:20.596332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: warnings.warn(msg) Jun 16 16:12:20.596332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: Jun 16 16:12:20.597767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1424a4-77f0-46b6-814a-5be6e66131c5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.597886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.597988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1424a4-77f0-46b6-814a-5be6e66131c5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.597988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1424a4-77f0-46b6-814a-5be6e66131c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.618666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 790/1571] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/users => generated 291 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.619497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.629542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa46b62-30b7-43bf-8c26-64e35bfb1acc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.630038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cc1424a4-77f0-46b6-814a-5be6e66131c5 None None] MFA Rules not processed for user `b2757df3207a404a834babbf189fdf95`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:20.631301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc1424a4-77f0-46b6-814a-5be6e66131c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:20.691322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.692062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.692346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.692672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.693355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:20.693697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.695404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.701781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 782/1572] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:20.702477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1424a4-77f0-46b6-814a-5be6e66131c5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.719971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 791/1573] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:20.719971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.720544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1424a4-77f0-46b6-814a-5be6e66131c5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.720544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.720544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.720544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.720910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.720910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.722632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0b73d-aa90-4a40-9a77-a0b36ca55085 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.722765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.727655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.728315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.728598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.728876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.729556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.732017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.751188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 783/1574] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.752251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.753616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 792/1575] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.754424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.757462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:20.766050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531776f-4851-452f-a5c3-b7cdfc92fc69 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.779427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:20.822768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa883b48-6d27-403c-a087-1a343a165b14 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.828765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.829444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.829742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.830153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.830846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.832785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.839946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.840661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.840943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.841236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.841990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.843935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.863025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 784/1576] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.864009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.878688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 793/1577] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.878688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.932804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca830f0e-33e8-4116-9353-fe8ff9ac9f36 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.938062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.938730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.939014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.939293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.939980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.941954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.942264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8b46b-6ba5-4ffe-ac98-62e7f7dbd3c7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.949011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.950420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.950826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.951217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.952279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.954925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.964955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 785/1578] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:20.965883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.966923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 794/1579] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:20.967672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:20.976545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084baadf-5a3a-4d03-9433-cde1d9751fd7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.979035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec948d7-4acf-4aef-b9ce-c49438048619 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:20.981125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.981816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.982102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.982379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.983092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.983871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:20.984634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:20.984986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:20.985118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:20.985770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:20.985925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:20.987772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.021694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 786/1580] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 357 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.021694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.024858 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 795/1581] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:20 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.025936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.087030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:21.092260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91b8bd-33e4-466c-8667-805107e98db5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.093130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70c19d-8fea-4728-8253-4c9661e26c44 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.097037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.097706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.098017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.098295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.098772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.098975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.099435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.099717 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.099991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.100713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.100928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.102613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.113916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 787/1582] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.114917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.124850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 796/1583] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.125672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.126755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b82666-7ba6-401b-ad1d-7e9682ada1e9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.131642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.132606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.132727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.133005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.133692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.135663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.136062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a5586-6282-41aa-b915-912b8d5ba1e8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.141559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.142370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.142654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.142896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.143666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.145525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.154285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:21.160529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 788/1584] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.162704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.172424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 797/1585] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/users => generated 303 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.173326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101956b-f1aa-4a4a-b6e5-27eeefdaa655 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.174451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.181059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.181789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.182082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.182359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.183088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.184746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.186923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72ccb4-d26d-4ee1-abe5-b8bb784dfc16 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.191557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.192224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.192513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.192795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.193521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.195159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.196027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 789/1586] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.196579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.207579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10642118-3ca6-4c5d-9bf5-e50cfce240f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.210226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 798/1587] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.211140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.213541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.214425 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.214703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.215050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.216538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.219194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.222554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ca139-1a77-4382-bcbf-dba5418b4387 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.227217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.227895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.228187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.228473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.229159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.232144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.241791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 790/1588] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/users => generated 285 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.242623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.253655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1aa1a5-41cb-4901-aa6f-d5ef97709afc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.259425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 799/1589] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.259425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.261865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.263078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.263363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.263689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.264423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.266089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.269845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da83e7-1a51-48ca-afad-6d46e97de032 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.274666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.275340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.275635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.275915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.276636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.277266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 791/1590] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.278320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.278449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.288713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae1fc9-2c35-4bdc-97f7-e82afeb95737 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.290334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 800/1591] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.291143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.294163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.294847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.295136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.295419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.296152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.297830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.301439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca30db-3f38-4a0c-854f-71e9711e9341 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.306256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.306922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.307205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.307481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.308204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.310357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.310522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 792/1592] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.311633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.321191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 801/1593] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.321969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cc1b2-4e5e-4e3c-808d-47cedf9b7524 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.324923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.326999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.327765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.328087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.328366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.329100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.330760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.335086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5249e-9ad2-4182-b46f-9080312541ee admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.341499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.342240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 793/1594] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.342481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.342775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.343068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] PATH_INFO: `/v3/projects/2bd59c3bdfc642558406ac4d6904d69a/users/64ae94dc289c4c34b14c0496003311e8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.343196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.354773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16883f2d-156f-49a7-bf72-9f37be2a66dc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.360370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.361031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.361316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.361594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.362163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd59c3bdfc642558406ac4d6904d69a, user_id=64ae94dc289c4c34b14c0496003311e8, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.362361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.363946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.364919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.375733 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 794/1595] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.378612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.380937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 802/1596] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/2bd59c3bdfc642558406ac4d6904d69a/users/64ae94dc289c4c34b14c0496003311e8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:21.391306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.391409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5d2cb-9ae1-407c-a0c9-a7f4aaf464a2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.446546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f28440-4cea-4ed0-a32c-24deb6acdb2b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.451452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.451923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.452134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.452425 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.452726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.452837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.453070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.453347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] PATH_INFO: `/v3/projects/dbfb390720a54a4eb35c80b0d1a1f16b/users/46445560ea7e496fa6c9c2e7c9a5b747/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.453455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.455162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.465351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 795/1597] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.469260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbfb390720a54a4eb35c80b0d1a1f16b, user_id=46445560ea7e496fa6c9c2e7c9a5b747, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.469959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.472210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.480831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cfc61-7bbe-40bc-97a3-13451c661541 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.485389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 803/1598] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/dbfb390720a54a4eb35c80b0d1a1f16b/users/46445560ea7e496fa6c9c2e7c9a5b747/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:21.486224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.532765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:21.539702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.540448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.540915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.541204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.542117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.544408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.548457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:21.553299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abd34-6586-4ba9-9b03-7d7c075899ab admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.554757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 796/1599] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.558405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.560128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.560982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.561272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.561556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.562314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.563927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.568728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda01086-512c-426d-967d-8fe5df795bae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.573280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.573959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.574147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 804/1600] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.574521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.574885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.574983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.575552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.577158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.584611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09945455-e8e2-4dbd-96f1-b9abc3ed8140 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.591306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 797/1601] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.591306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.592221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73f099-de70-47dd-a044-217535b20978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.593074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73f099-de70-47dd-a044-217535b20978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.593360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73f099-de70-47dd-a044-217535b20978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.593655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73f099-de70-47dd-a044-217535b20978 admin admin] PATH_INFO: `/v3/projects/c3592ee016d1486db983b4ff2aab5ca3/users/16b8389777744e6581ad14be85731f4a/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.602185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9af17-52c7-44ad-bb75-fd220e8d1e62 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.607139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.608033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.608349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.608632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] PATH_INFO: `/v3/projects/290e7c15bbd84633872b63e39ab192f0/users/f8781ce43cb54045b5929d94faf74b51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.609822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73f099-de70-47dd-a044-217535b20978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3592ee016d1486db983b4ff2aab5ca3, user_id=16b8389777744e6581ad14be85731f4a, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.613613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73f099-de70-47dd-a044-217535b20978 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.625390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] RBAC: Authorizing `identity:create_grant(project_id=290e7c15bbd84633872b63e39ab192f0, user_id=f8781ce43cb54045b5929d94faf74b51, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.626578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:12:21.626578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:12:21.626578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:12:21.628399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 805/1602] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/c3592ee016d1486db983b4ff2aab5ca3/users/16b8389777744e6581ad14be85731f4a/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:21.628561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.629706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12195748-63f9-4ba7-86bf-babc841f3429 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.630024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12195748-63f9-4ba7-86bf-babc841f3429 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.630303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12195748-63f9-4ba7-86bf-babc841f3429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.643413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 798/1603] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/290e7c15bbd84633872b63e39ab192f0/users/f8781ce43cb54045b5929d94faf74b51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:21.644195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.658135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-12195748-63f9-4ba7-86bf-babc841f3429 None None] MFA Rules not processed for user `64ae94dc289c4c34b14c0496003311e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:21.659378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-12195748-63f9-4ba7-86bf-babc841f3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:21.708861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cb346-2ef8-4339-a1ed-5f81f54b323f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.713806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.714367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.714656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.714936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.716311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.717817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.727411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 806/1604] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:21.731628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 799/1605] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.731803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ecf42-de11-49f3-a3de-33ca95b8a742 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.731892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.731971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ecf42-de11-49f3-a3de-33ca95b8a742 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.731971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ecf42-de11-49f3-a3de-33ca95b8a742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.741041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a019ce1-85ee-4abf-8f0d-6d4473ddf833 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.745712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.746280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-714ecf42-de11-49f3-a3de-33ca95b8a742 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:21.746621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.747438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.747717 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] PATH_INFO: `/v3/projects/c1cdf363c2e44af5b0d8cd748c58235b/users/bfbcc9b45bc341228bcc0f2192040224/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.747992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-714ecf42-de11-49f3-a3de-33ca95b8a742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:21.756880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 807/1606] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:21.760570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ecf42-de11-49f3-a3de-33ca95b8a742 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.766970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1cdf363c2e44af5b0d8cd748c58235b, user_id=bfbcc9b45bc341228bcc0f2192040224, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.768442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ecf42-de11-49f3-a3de-33ca95b8a742 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.769671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.773009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341e301-26a8-4831-b6c1-a45169897dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.774084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e301-26a8-4831-b6c1-a45169897dce admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.774293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e301-26a8-4831-b6c1-a45169897dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.774663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e301-26a8-4831-b6c1-a45169897dce admin admin] PATH_INFO: `/v3/projects/24fb126babee4bbea59923603503a21a/users/5e2021bba1874f0fb4e6904aac576431/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.784221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 800/1607] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/c1cdf363c2e44af5b0d8cd748c58235b/users/bfbcc9b45bc341228bcc0f2192040224/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:21.785086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.791106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341e301-26a8-4831-b6c1-a45169897dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fb126babee4bbea59923603503a21a, user_id=5e2021bba1874f0fb4e6904aac576431, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.794342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341e301-26a8-4831-b6c1-a45169897dce admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.809121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 808/1608] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/24fb126babee4bbea59923603503a21a/users/5e2021bba1874f0fb4e6904aac576431/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:21.811611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86134c0a-46fe-4ec5-a168-01077a13d9ce None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.811905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86134c0a-46fe-4ec5-a168-01077a13d9ce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.812186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86134c0a-46fe-4ec5-a168-01077a13d9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.829612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-86134c0a-46fe-4ec5-a168-01077a13d9ce None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:21.830911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-86134c0a-46fe-4ec5-a168-01077a13d9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:21.853348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277db56-5913-4473-9313-3ed2e6f400be admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.887108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:21.893206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 809/1609] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:21.894184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86134c0a-46fe-4ec5-a168-01077a13d9ce None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.904445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86134c0a-46fe-4ec5-a168-01077a13d9ce None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.909031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.909738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.910032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.910310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.911028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.912616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.915280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.916246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.916612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.917076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] PATH_INFO: `/v3/projects/125583532148403bad4a146c2d3be672/users/5fcb3530bbd84c219c76e1f3005546a8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.923390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 810/1610] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:21.924170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.933597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274f1eb-f4de-4ee0-b04d-2c4c1a2c3daf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:21.934963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=125583532148403bad4a146c2d3be672, user_id=5fcb3530bbd84c219c76e1f3005546a8, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.937773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.938775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:21.939421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:21.939703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:21.939983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:21.940661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:21.942585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:21.951905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 801/1611] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/125583532148403bad4a146c2d3be672/users/5fcb3530bbd84c219c76e1f3005546a8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:21.953332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:21.959162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:21.973453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 811/1612] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:21 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:21.974848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.019590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad44aa0-0605-449e-a569-b0151a13a695 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.031329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e80ac72-18e2-4fc0-8c7b-fff7136ebb5f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.035243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53255c-8d1b-4c3a-b3a8-b9e215b1815f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.035891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53255c-8d1b-4c3a-b3a8-b9e215b1815f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.036175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53255c-8d1b-4c3a-b3a8-b9e215b1815f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.036576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53255c-8d1b-4c3a-b3a8-b9e215b1815f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.078173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b3eda-7064-41e6-90f9-6109da8452b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.079260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3eda-7064-41e6-90f9-6109da8452b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.079598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3eda-7064-41e6-90f9-6109da8452b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.079940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b3eda-7064-41e6-90f9-6109da8452b2 admin admin] PATH_INFO: `/v3/projects/391132d6e00c4f04b438e886be220105/users/7ec9fb97bb154c41b6f0f2bfe1b9e7e9/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.095659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3eda-7064-41e6-90f9-6109da8452b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391132d6e00c4f04b438e886be220105, user_id=7ec9fb97bb154c41b6f0f2bfe1b9e7e9, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.098401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b3eda-7064-41e6-90f9-6109da8452b2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.104381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53255c-8d1b-4c3a-b3a8-b9e215b1815f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.105932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53255c-8d1b-4c3a-b3a8-b9e215b1815f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.114092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 802/1613] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:21 2022] PUT /identity/v3/projects/391132d6e00c4f04b438e886be220105/users/7ec9fb97bb154c41b6f0f2bfe1b9e7e9/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:22.115510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd49b6-5c00-468d-b49c-66bfe5157982 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.115803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd49b6-5c00-468d-b49c-66bfe5157982 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.116091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd49b6-5c00-468d-b49c-66bfe5157982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.116369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 812/1614] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:22.117648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2a336-62c8-4f4e-b043-7b1a03ee6ec0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.117972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2a336-62c8-4f4e-b043-7b1a03ee6ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.118255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2a336-62c8-4f4e-b043-7b1a03ee6ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.137780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e0cd49b6-5c00-468d-b49c-66bfe5157982 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:22.138370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e0cd49b6-5c00-468d-b49c-66bfe5157982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.142635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-29a2a336-62c8-4f4e-b043-7b1a03ee6ec0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:22.143882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-29a2a336-62c8-4f4e-b043-7b1a03ee6ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.205615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:22.223937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 813/1615] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:22.223937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2a336-62c8-4f4e-b043-7b1a03ee6ec0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.234224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 803/1616] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:22.235076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd49b6-5c00-468d-b49c-66bfe5157982 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.245520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd49b6-5c00-468d-b49c-66bfe5157982 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.250780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52deda-ef4a-4d4a-8b91-93af0d1656ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.251704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52deda-ef4a-4d4a-8b91-93af0d1656ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.252041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52deda-ef4a-4d4a-8b91-93af0d1656ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.252329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52deda-ef4a-4d4a-8b91-93af0d1656ac admin admin] PATH_INFO: `/v3/projects/290e7c15bbd84633872b63e39ab192f0/users/f8781ce43cb54045b5929d94faf74b51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.254544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52deda-ef4a-4d4a-8b91-93af0d1656ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=290e7c15bbd84633872b63e39ab192f0, user_id=f8781ce43cb54045b5929d94faf74b51, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.257274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52deda-ef4a-4d4a-8b91-93af0d1656ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.270110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 804/1617] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:22 2022] PUT /identity/v3/projects/290e7c15bbd84633872b63e39ab192f0/users/f8781ce43cb54045b5929d94faf74b51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:22.271438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68a09b-cf21-429e-a5d6-281a60403d3f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.271732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68a09b-cf21-429e-a5d6-281a60403d3f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.272032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68a09b-cf21-429e-a5d6-281a60403d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.293005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d68a09b-cf21-429e-a5d6-281a60403d3f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:22.294400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d68a09b-cf21-429e-a5d6-281a60403d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.301368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:22.307756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2a336-62c8-4f4e-b043-7b1a03ee6ec0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.381513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61be83-4443-483b-831b-f071666f63ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.382485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61be83-4443-483b-831b-f071666f63ec service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.382785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 805/1618] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:22.383605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68a09b-cf21-429e-a5d6-281a60403d3f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.385985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61be83-4443-483b-831b-f071666f63ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.386466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61be83-4443-483b-831b-f071666f63ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.399203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61be83-4443-483b-831b-f071666f63ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.399441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68a09b-cf21-429e-a5d6-281a60403d3f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.400835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61be83-4443-483b-831b-f071666f63ec service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.405291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.406067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.406359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.406712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.407617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.409600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.412671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 814/1619] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:22.414101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b3ffb-8914-4847-b2a5-cb2bae5cce08 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.414642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b3ffb-8914-4847-b2a5-cb2bae5cce08 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.414642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b3ffb-8914-4847-b2a5-cb2bae5cce08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.431450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 806/1620] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/users => generated 296 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:22.432119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.433793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.446685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340a353-b2ed-4361-9ba2-91bd33d4b2bc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.451065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-794b3ffb-8914-4847-b2a5-cb2bae5cce08 None None] MFA Rules not processed for user `7ec9fb97bb154c41b6f0f2bfe1b9e7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:22.451629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.452332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.452470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-794b3ffb-8914-4847-b2a5-cb2bae5cce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.452646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.452901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.464441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.466980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.478163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 807/1621] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:22.479378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.490112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1931c1-7fb0-4306-9883-08a79448341a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.501107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.501811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.502100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.502382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.517131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.518793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.533947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 815/1622] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:22.536158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531971b6-8533-49b8-b6ab-913c1afc29ed None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.537301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531971b6-8533-49b8-b6ab-913c1afc29ed None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.538162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531971b6-8533-49b8-b6ab-913c1afc29ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.541639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 808/1623] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:22.542997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.554297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042d1da-7f0a-4018-bcba-af8d25d60f31 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.560151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.560918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.561758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.562016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.574176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.574718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-531971b6-8533-49b8-b6ab-913c1afc29ed None None] MFA Rules not processed for user `f8781ce43cb54045b5929d94faf74b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:22.576204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.576628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-531971b6-8533-49b8-b6ab-913c1afc29ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.593112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 809/1624] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:22.594499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.607742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b30c7-11e5-43e4-88fe-9d282d66ce0d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.613750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b19fd4-1611-4deb-b873-48bab16ce575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.614701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b19fd4-1611-4deb-b873-48bab16ce575 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.615126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b19fd4-1611-4deb-b873-48bab16ce575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.615557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b19fd4-1611-4deb-b873-48bab16ce575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.616545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b19fd4-1611-4deb-b873-48bab16ce575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.619186 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b19fd4-1611-4deb-b873-48bab16ce575 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.636227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 810/1625] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:22.636227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d1f4-5a82-44b1-82de-118eb5b13abd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.636227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d1f4-5a82-44b1-82de-118eb5b13abd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.636227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d1f4-5a82-44b1-82de-118eb5b13abd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.639217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 811/1626] 10.222.0.88 () {54 vars in 961 bytes} [Thu Jun 16 16:12:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:22.640198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d1f4-5a82-44b1-82de-118eb5b13abd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.665821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d1f4-5a82-44b1-82de-118eb5b13abd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.666755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 816/1627] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:22.668274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4465b-a2e6-4ecc-a880-eef2c881c017 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.668821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4465b-a2e6-4ecc-a880-eef2c881c017 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.669663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4465b-a2e6-4ecc-a880-eef2c881c017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.689087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.690080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.690366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.690644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] PATH_INFO: `/v3/projects/bcb7f8a3c5324657b37ba4456dbd853a/users/38d2b3e641fd40d29aca3be5b6f82427/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.692628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e7d4465b-a2e6-4ecc-a880-eef2c881c017 None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:22.694212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e7d4465b-a2e6-4ecc-a880-eef2c881c017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:22.714446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:22.721055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb7f8a3c5324657b37ba4456dbd853a, user_id=38d2b3e641fd40d29aca3be5b6f82427, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.727329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.750012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 812/1628] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:22 2022] PUT /identity/v3/projects/bcb7f8a3c5324657b37ba4456dbd853a/users/38d2b3e641fd40d29aca3be5b6f82427/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:22.750012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.816099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 817/1629] 10.222.0.88 () {58 vars in 1058 bytes} [Thu Jun 16 16:12:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:22.817094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4465b-a2e6-4ecc-a880-eef2c881c017 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.831440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:22.834948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4465b-a2e6-4ecc-a880-eef2c881c017 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.839066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c51a0d-490b-42a8-9052-226c1ecead31 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:22.846284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.847103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.847784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.848433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.937065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.938587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.948083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:22.949412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:22.949855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:22.950248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:22.951319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:22.954713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:22.962475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 813/1630] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:22.965935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:22.973551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 818/1631] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:22.977127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:23.006409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376430ff-4bcd-4b8d-88ed-f5f0c694a026 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:23.031669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465539b-217a-4813-9aa7-6442a1a4a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.032919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465539b-217a-4813-9aa7-6442a1a4a9bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.034058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465539b-217a-4813-9aa7-6442a1a4a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.035793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465539b-217a-4813-9aa7-6442a1a4a9bd admin admin] PATH_INFO: `/v3/projects/bcb7f8a3c5324657b37ba4456dbd853a/users/38d2b3e641fd40d29aca3be5b6f82427/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.041506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465539b-217a-4813-9aa7-6442a1a4a9bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb7f8a3c5324657b37ba4456dbd853a, user_id=38d2b3e641fd40d29aca3be5b6f82427, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:23.049985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465539b-217a-4813-9aa7-6442a1a4a9bd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.083806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 819/1632] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:22 2022] PUT /identity/v3/projects/bcb7f8a3c5324657b37ba4456dbd853a/users/38d2b3e641fd40d29aca3be5b6f82427/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:23.099261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93936f-b800-4f25-9805-ed70ee1a1569 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.099550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93936f-b800-4f25-9805-ed70ee1a1569 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.100019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93936f-b800-4f25-9805-ed70ee1a1569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.118502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94721dca-99b3-4b4d-b6c3-bf2cc4aa381d service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:23.134839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8d93936f-b800-4f25-9805-ed70ee1a1569 None None] MFA Rules not processed for user `38d2b3e641fd40d29aca3be5b6f82427`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:23.136145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8d93936f-b800-4f25-9805-ed70ee1a1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:23.259918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.261537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.262737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.263255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.269849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 820/1633] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:23.273163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93936f-b800-4f25-9805-ed70ee1a1569 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:23.308602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:23.319850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:23.383645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93936f-b800-4f25-9805-ed70ee1a1569 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:23.390905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb0f403-2171-40e3-a787-b85c9619f15f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.391734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0f403-2171-40e3-a787-b85c9619f15f service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.392129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0f403-2171-40e3-a787-b85c9619f15f service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.392648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0f403-2171-40e3-a787-b85c9619f15f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.409772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:23.414479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.444597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 814/1634] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:23.447063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:23.466272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3ddf9-1bce-410d-afba-14ac4db145c3 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:23.476055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d61bd8-9811-4a1f-83ca-a10532d8d4db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:23.478197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d61bd8-9811-4a1f-83ca-a10532d8d4db service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.478489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d61bd8-9811-4a1f-83ca-a10532d8d4db service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.478770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d61bd8-9811-4a1f-83ca-a10532d8d4db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.503284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d61bd8-9811-4a1f-83ca-a10532d8d4db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:23.505023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d61bd8-9811-4a1f-83ca-a10532d8d4db service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.531528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 815/1635] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:23.534805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0f403-2171-40e3-a787-b85c9619f15f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:23.536256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb0f403-2171-40e3-a787-b85c9619f15f service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:23.547287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 821/1636] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:23 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:23.952011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad154e-fdae-4959-b8bf-10f5df381021 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.952624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad154e-fdae-4959-b8bf-10f5df381021 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.952624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad154e-fdae-4959-b8bf-10f5df381021 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:23.955122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 816/1637] 10.222.0.88 () {54 vars in 912 bytes} [Thu Jun 16 16:12:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:23.972787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d243-2b8b-43c3-a9ff-a746f3ded6b9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:23.973394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d243-2b8b-43c3-a9ff-a746f3ded6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:23.973394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d243-2b8b-43c3-a9ff-a746f3ded6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.031283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2514d243-2b8b-43c3-a9ff-a746f3ded6b9 None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:24.032524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2514d243-2b8b-43c3-a9ff-a746f3ded6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:24.063300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 822/1638] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:12:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:24.098097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad154e-fdae-4959-b8bf-10f5df381021 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:24.118184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad154e-fdae-4959-b8bf-10f5df381021 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:24.143405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:24.144201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.144487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.145591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.163292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.164536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.186787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 817/1639] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:24.207367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514d243-2b8b-43c3-a9ff-a746f3ded6b9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:24.224902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514d243-2b8b-43c3-a9ff-a746f3ded6b9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:24.238437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d2b3e641fd40d29aca3be5b6f82427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcb7f8a3c5324657b37ba4456dbd853a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:24.239297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.239426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.239682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.241034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:24.241683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.245014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.290055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 823/1640] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:24.291912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:24.310517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7e230-f9f6-4cc9-b956-6dce8f207b15 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:24.325914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d2b3e641fd40d29aca3be5b6f82427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcb7f8a3c5324657b37ba4456dbd853a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:24.327161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.327440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.327832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.328528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.332770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.399594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 818/1641] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:24 2022] POST /identity/v3/projects => generated 353 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:24.401560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:24.417668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5f5ff-cd3d-4cbf-aa1a-f03e6ccad935 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:24.565788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81db3a-758b-4550-884a-4202f1ca0e17 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d2b3e641fd40d29aca3be5b6f82427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcb7f8a3c5324657b37ba4456dbd853a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:24.566509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81db3a-758b-4550-884a-4202f1ca0e17 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.566809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81db3a-758b-4550-884a-4202f1ca0e17 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.567097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81db3a-758b-4550-884a-4202f1ca0e17 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.567845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81db3a-758b-4550-884a-4202f1ca0e17 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:24.570295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81db3a-758b-4550-884a-4202f1ca0e17 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.578688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81db3a-758b-4550-884a-4202f1ca0e17 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.631409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 824/1642] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:24.685950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:24.707435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c863bd7-cc9d-464a-b187-efca99472b32 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:24.717485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d2b3e641fd40d29aca3be5b6f82427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcb7f8a3c5324657b37ba4456dbd853a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:24.719016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.724297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.725369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] PATH_INFO: `/v3/projects/71ce45653bb941d2900b7281efe5f7c6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.729816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorizing `identity:delete_project(project_id=71ce45653bb941d2900b7281efe5f7c6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:24.732412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:24.827464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ce45653bb941d2900b7281efe5f7c6', 'request_id': 'req-b580d352-ddde-4cbb-a6d4-9916d64c4db4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.849547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ce45653bb941d2900b7281efe5f7c6', 'request_id': 'req-b580d352-ddde-4cbb-a6d4-9916d64c4db4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.862174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ce45653bb941d2900b7281efe5f7c6', 'request_id': 'req-b580d352-ddde-4cbb-a6d4-9916d64c4db4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.886257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] The token cache is being invalidate because project 71ce45653bb941d2900b7281efe5f7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:24.886591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.887634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:24.896307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 819/1643] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:24 2022] DELETE /identity/v3/projects/71ce45653bb941d2900b7281efe5f7c6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:24.940696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:24.941154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:24.941491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:24.981897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] MFA Rules not processed for user `5fcb3530bbd84c219c76e1f3005546a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:24.983930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.050422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:25.174460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 825/1644] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:24 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:25.178704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.180357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.396449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580d352-ddde-4cbb-a6d4-9916d64c4db4 tempest-VolumeQuotaClassesTest-770223132 tempest-VolumeQuotaClassesTest-770223132-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.409219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.411331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.411733 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.412419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.414774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.421079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.431435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.433354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.441659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:25.462945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aac490-e0f8-4384-8454-a9241361a794 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.466776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 820/1645] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/projects => generated 405 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:25.471234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.632487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.633692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.633991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.634266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] PATH_INFO: `/v3/users/64ae94dc289c4c34b14c0496003311e8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.636084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ae94dc289c4c34b14c0496003311e8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.638838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.686551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e8367-6821-4ff0-aeb3-5e18a669c04b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.696921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.698380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.699187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.700397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.701931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.704459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.785489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 821/1646] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/users => generated 309 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:25.787466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.791687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ae94dc289c4c34b14c0496003311e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:25.793947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ae94dc289c4c34b14c0496003311e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:25.805813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f45eb5-faa8-4d65-b24e-8e4113e8c891 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.814253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.815671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ae94dc289c4c34b14c0496003311e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:25.815859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.816364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.818048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.819910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ae94dc289c4c34b14c0496003311e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:25.831578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.833602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.840880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-15d392f1-5b97-4a8a-b98a-feac49b72c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ae94dc289c4c34b14c0496003311e8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:25.855705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 822/1647] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:25.858362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:25.860644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:25.880533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67410c-0cfe-43b2-874f-c7e71caf9d2c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:25.885771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 826/1648] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:25 2022] DELETE /identity/v3/users/64ae94dc289c4c34b14c0496003311e8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:25.892098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d130e0-b308-42c4-94d8-77cf53574a14 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.892419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d130e0-b308-42c4-94d8-77cf53574a14 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.892669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d130e0-b308-42c4-94d8-77cf53574a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.894298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861252ae-7076-48c3-b6e4-c344236b7ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:25.895213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861252ae-7076-48c3-b6e4-c344236b7ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:25.895591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861252ae-7076-48c3-b6e4-c344236b7ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:25.895750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861252ae-7076-48c3-b6e4-c344236b7ade admin admin] PATH_INFO: `/v3/projects/bfa27850e0d247a982554a2065e54f43/users/7dd332d52c9b47fe9cf9ffa1b33722e1/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:25.926646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861252ae-7076-48c3-b6e4-c344236b7ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa27850e0d247a982554a2065e54f43, user_id=7dd332d52c9b47fe9cf9ffa1b33722e1, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:25.931520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861252ae-7076-48c3-b6e4-c344236b7ade admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:25.943258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a0d130e0-b308-42c4-94d8-77cf53574a14 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:25.946816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a0d130e0-b308-42c4-94d8-77cf53574a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:26.072783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 823/1649] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:25 2022] PUT /identity/v3/projects/bfa27850e0d247a982554a2065e54f43/users/7dd332d52c9b47fe9cf9ffa1b33722e1/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:26.072783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951475-d3d1-48b6-981f-8ade8e5a107e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.072783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951475-d3d1-48b6-981f-8ade8e5a107e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.072783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f951475-d3d1-48b6-981f-8ade8e5a107e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.072783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 824/1650] 10.222.0.88 () {54 vars in 961 bytes} [Thu Jun 16 16:12:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:26.072783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f951475-d3d1-48b6-981f-8ade8e5a107e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.073452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:26.084627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 827/1651] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:26.089976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff8d1-c681-4f82-9556-61b142462362 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.090275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff8d1-c681-4f82-9556-61b142462362 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.091153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eff8d1-c681-4f82-9556-61b142462362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.141951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-65eff8d1-c681-4f82-9556-61b142462362 None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:26.142847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-65eff8d1-c681-4f82-9556-61b142462362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:26.240308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f951475-d3d1-48b6-981f-8ade8e5a107e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.255271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.256146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.256537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.256917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.258030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.260739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.289381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 825/1652] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:26.290525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.294583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:26.349437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 828/1653] 10.222.0.88 () {58 vars in 1058 bytes} [Thu Jun 16 16:12:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:26.351848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eff8d1-c681-4f82-9556-61b142462362 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.377339 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eff8d1-c681-4f82-9556-61b142462362 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.389077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68c118-bac1-4fb2-8468-8d6d5166908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.394392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68c118-bac1-4fb2-8468-8d6d5166908d admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.394720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68c118-bac1-4fb2-8468-8d6d5166908d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.395154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68c118-bac1-4fb2-8468-8d6d5166908d admin admin] PATH_INFO: `/v3/projects/bfa27850e0d247a982554a2065e54f43/users/7dd332d52c9b47fe9cf9ffa1b33722e1/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.398711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68c118-bac1-4fb2-8468-8d6d5166908d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa27850e0d247a982554a2065e54f43, user_id=7dd332d52c9b47fe9cf9ffa1b33722e1, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.402217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:26.403620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68c118-bac1-4fb2-8468-8d6d5166908d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.432804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ff157-39ee-4816-8540-8c6041737cb8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:26.436799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 829/1654] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:26 2022] PUT /identity/v3/projects/bfa27850e0d247a982554a2065e54f43/users/7dd332d52c9b47fe9cf9ffa1b33722e1/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:26.441416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7358-c640-439c-99dd-84d0da79cacb None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.441416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7358-c640-439c-99dd-84d0da79cacb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.441416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7358-c640-439c-99dd-84d0da79cacb None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.441416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 830/1655] 10.222.0.88 () {54 vars in 912 bytes} [Thu Jun 16 16:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:26.458012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.458304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.458587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.506402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:26.507876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:26.563409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:26.564780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:26.567102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:26.568323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:26.716290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:26.718038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:26.749309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 831/1656] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:12:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:26.765764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 826/1657] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:26.806731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:26.817300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:27.034970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fb4a5-2647-4d21-bae8-dc950ff3cd1c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:27.049395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:27.050231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:27.050638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:27.051895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:27.078747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:27.080244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:27.108394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 832/1658] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:28.180043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:28.181546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:28.203402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef90d36-752b-453d-bce9-ced8645d860a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:28.217182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:28.218150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:28.218403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:28.218681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] PATH_INFO: `/v3/projects/2bd59c3bdfc642558406ac4d6904d69a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:28.219959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd59c3bdfc642558406ac4d6904d69a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:28.222266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:28.328203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd59c3bdfc642558406ac4d6904d69a', 'request_id': 'req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.347224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd59c3bdfc642558406ac4d6904d69a', 'request_id': 'req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.430654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd59c3bdfc642558406ac4d6904d69a', 'request_id': 'req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.467794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] The token cache is being invalidate because project 2bd59c3bdfc642558406ac4d6904d69a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:28.468344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.469637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.472261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 827/1659] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:28 2022] DELETE /identity/v3/projects/2bd59c3bdfc642558406ac4d6904d69a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:28.476998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:28.491793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a7d88-675b-40ad-b16a-2d30a7656863 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:28.501854 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:28.618862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:28.620996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:28.622769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:28.623169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] PATH_INFO: `/v3/users/38d2b3e641fd40d29aca3be5b6f82427` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:28.625909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d2b3e641fd40d29aca3be5b6f82427)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:28.632629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:28.755689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d2b3e641fd40d29aca3be5b6f82427'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.756547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d2b3e641fd40d29aca3be5b6f82427'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.794476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d2b3e641fd40d29aca3be5b6f82427'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.796204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d2b3e641fd40d29aca3be5b6f82427'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.823930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-879b5b58-96b6-4aa9-aa21-771aed241e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d2b3e641fd40d29aca3be5b6f82427'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:28.867937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 833/1660] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:28 2022] DELETE /identity/v3/users/38d2b3e641fd40d29aca3be5b6f82427 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:29.070080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:29.071576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:29.108318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aaa6a-64c2-402f-9f58-13ede4a1fd3e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:29.210713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:29.294761 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:29.297479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:29.300297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:29.300727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] PATH_INFO: `/v3/users/7ec9fb97bb154c41b6f0f2bfe1b9e7e9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:29.303486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec9fb97bb154c41b6f0f2bfe1b9e7e9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:29.304739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:12:29.304739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) Jun 16 16:12:29.304739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: Jun 16 16:12:29.307596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:29.472653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec9fb97bb154c41b6f0f2bfe1b9e7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:29.473388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec9fb97bb154c41b6f0f2bfe1b9e7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:29.474070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec9fb97bb154c41b6f0f2bfe1b9e7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:29.489228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec9fb97bb154c41b6f0f2bfe1b9e7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:29.503557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec9fb97bb154c41b6f0f2bfe1b9e7e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:29.557478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 828/1661] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:29 2022] DELETE /identity/v3/users/7ec9fb97bb154c41b6f0f2bfe1b9e7e9 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:29.563110 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b67a5-65cf-488f-90bb-386201d504d6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:29.564051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b67a5-65cf-488f-90bb-386201d504d6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:29.564714 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b67a5-65cf-488f-90bb-386201d504d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:29.590366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7a6b67a5-65cf-488f-90bb-386201d504d6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:29.592348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7a6b67a5-65cf-488f-90bb-386201d504d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:29.767340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 834/1662] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:29.823872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:29.826696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:30.113866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb027d51-afab-4549-92cf-d9f444c4983e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:30.122073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c4bda-6dcf-45be-826a-3e7ad31bc3ae None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:30.122559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c4bda-6dcf-45be-826a-3e7ad31bc3ae None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:30.124091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c4bda-6dcf-45be-826a-3e7ad31bc3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:30.149936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:30.150899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:30.151195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:30.151904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:30.201854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:30.204469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:30.218439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-230c4bda-6dcf-45be-826a-3e7ad31bc3ae None None] MFA Rules not processed for user `7dd332d52c9b47fe9cf9ffa1b33722e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:30.228595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-230c4bda-6dcf-45be-826a-3e7ad31bc3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:30.305301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 829/1663] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:30.414410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:30.593417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:30.594944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:30.598803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 835/1664] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:30.622523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72c294-2e38-46bf-a5c6-db064b52a8d1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:30.649765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:30.656301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:30.656606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:30.656896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] PATH_INFO: `/v3/projects/bcb7f8a3c5324657b37ba4456dbd853a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:30.668587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb7f8a3c5324657b37ba4456dbd853a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:30.671215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:30.818630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:30.821520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb7f8a3c5324657b37ba4456dbd853a', 'request_id': 'req-d3ffcca2-9e64-4770-8bd7-21194532f434'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:30.838517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb7f8a3c5324657b37ba4456dbd853a', 'request_id': 'req-d3ffcca2-9e64-4770-8bd7-21194532f434'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:30.859299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb7f8a3c5324657b37ba4456dbd853a', 'request_id': 'req-d3ffcca2-9e64-4770-8bd7-21194532f434'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:30.885217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] The token cache is being invalidate because project bcb7f8a3c5324657b37ba4456dbd853a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:30.885677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:30.886611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d3ffcca2-9e64-4770-8bd7-21194532f434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:30.888855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 830/1665] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:30 2022] DELETE /identity/v3/projects/bcb7f8a3c5324657b37ba4456dbd853a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:30.896399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d504139-2dbe-48da-b7a2-a7a721da9887 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:30.897530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d504139-2dbe-48da-b7a2-a7a721da9887 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:30.898027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d504139-2dbe-48da-b7a2-a7a721da9887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:30.943325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7d504139-2dbe-48da-b7a2-a7a721da9887 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:30.944679 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7d504139-2dbe-48da-b7a2-a7a721da9887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:31.124394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 836/1666] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:31.147448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d4e3c-6d57-4756-8f34-256dde7ca602 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.147448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d4e3c-6d57-4756-8f34-256dde7ca602 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.148406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d4e3c-6d57-4756-8f34-256dde7ca602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:31.211318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-572d4e3c-6d57-4756-8f34-256dde7ca602 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:31.213499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-572d4e3c-6d57-4756-8f34-256dde7ca602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:31.243182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 831/1667] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:31.247615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d504139-2dbe-48da-b7a2-a7a721da9887 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:31.266698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d504139-2dbe-48da-b7a2-a7a721da9887 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:31.288664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:31.289370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.296621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.296952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:31.298061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:31.298632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:31.300800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:31.370812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 837/1668] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:31.372660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d4e3c-6d57-4756-8f34-256dde7ca602 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:31.411301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d4e3c-6d57-4756-8f34-256dde7ca602 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:31.418629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:31.419613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.420041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.420470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:31.421889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:31.425306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:31.503974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 832/1669] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:31 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:31.508080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:31.537221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bab3f8-d753-4bfd-8683-c7a77fa1bee6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:31.665938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:31.667416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.668106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.669823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:31.670773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:31.672952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:31.720209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 838/1670] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:31 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:31.726016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:31.748785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71906e3-3fe8-48aa-9d3c-644c1cc4d508 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:31.773324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:31.775543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.775879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.776210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:31.787581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:31.789381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:31.804499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 833/1671] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:31.808553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:31.822239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8bcf9-f2de-4708-9b72-8c0efa2a1749 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:31.836588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb8e89-9416-44ca-9c1d-d30cef528c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:31.838182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb8e89-9416-44ca-9c1d-d30cef528c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.838512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb8e89-9416-44ca-9c1d-d30cef528c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.838749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb8e89-9416-44ca-9c1d-d30cef528c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:31.839538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb8e89-9416-44ca-9c1d-d30cef528c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:31.841355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb8e89-9416-44ca-9c1d-d30cef528c27 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:31.874031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 839/1672] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:31.878286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:31.892748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7136a-f6bc-4a91-85e5-64621b83620f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:31.900044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:31.901641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.902253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.903150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] PATH_INFO: `/v3/projects/5df9f0e03bc94dc9916183aee67492bd/users/8e7fbb5511824b549c3ef1e306138ade/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:31.931837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df9f0e03bc94dc9916183aee67492bd, user_id=8e7fbb5511824b549c3ef1e306138ade, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:31.935914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:31.958376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 834/1673] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:31 2022] PUT /identity/v3/projects/5df9f0e03bc94dc9916183aee67492bd/users/8e7fbb5511824b549c3ef1e306138ade/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:12:31.968419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7268-7a52-499b-b64e-d3b70ae233ce None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:31.968726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7268-7a52-499b-b64e-d3b70ae233ce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:31.969036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7268-7a52-499b-b64e-d3b70ae233ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:32.035055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ce5d7268-7a52-499b-b64e-d3b70ae233ce None None] MFA Rules not processed for user `8e7fbb5511824b549c3ef1e306138ade`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:32.038622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ce5d7268-7a52-499b-b64e-d3b70ae233ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:32.061784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:32.231866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 840/1674] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:32.236219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:32.437915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0784c4-a508-435e-b57f-d4d85903227a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:32.457326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:32.458421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:32.458719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:32.458999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:32.459762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:32.461756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:32.571202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 835/1675] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:32 2022] POST /identity/v3/projects => generated 363 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:32.577025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7268-7a52-499b-b64e-d3b70ae233ce None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:32.597888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7268-7a52-499b-b64e-d3b70ae233ce None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:32.775057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:32.776263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:32.776563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:32.776850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:32.788207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:32.790786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:32.865961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 841/1676] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:32 2022] POST /identity/v3/users => generated 288 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:32.869916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:32.896374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac8d0-36ff-40c4-b3be-0829494bb68b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:32.908357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:32.909057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:32.909343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:32.909628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:32.910508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:32.912194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:32.962341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 836/1677] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:32.967449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:32.984680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f3d2d-9a98-4d5c-865a-b9f0d484f300 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.009262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.010778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.010941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.011228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] PATH_INFO: `/v3/projects/5e7d43ccef4e4b9a9320735a98558707/users/d3fd07bd9b3b473ba7e7c40d57f44729/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.062357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7d43ccef4e4b9a9320735a98558707, user_id=d3fd07bd9b3b473ba7e7c40d57f44729, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:33.065543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:33.095999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 842/1678] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:32 2022] PUT /identity/v3/projects/5e7d43ccef4e4b9a9320735a98558707/users/d3fd07bd9b3b473ba7e7c40d57f44729/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:33.102149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:33.115754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07353a1c-b9dc-4a60-8fd6-c761a8361ae3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.269783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:33.303838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.305374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.305685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.305997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.306798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:33.312671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:33.342415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 837/1679] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:33.348080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:33.356248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:33.358267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.362386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2143-e230-40f4-b333-71b996dd4160 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.377019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528246ac-4443-4a4d-af88-a3244c21f437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.377911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a2f5c-8f88-451f-9cfb-9b2cb62e21ae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.378035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528246ac-4443-4a4d-af88-a3244c21f437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.378235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528246ac-4443-4a4d-af88-a3244c21f437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.378859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528246ac-4443-4a4d-af88-a3244c21f437 admin admin] PATH_INFO: `/v3/projects/5e7d43ccef4e4b9a9320735a98558707/users/d3fd07bd9b3b473ba7e7c40d57f44729/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.383417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528246ac-4443-4a4d-af88-a3244c21f437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7d43ccef4e4b9a9320735a98558707, user_id=d3fd07bd9b3b473ba7e7c40d57f44729, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:33.391418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528246ac-4443-4a4d-af88-a3244c21f437 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:33.391702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.405036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.407316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.407824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] PATH_INFO: `/v3/projects/391132d6e00c4f04b438e886be220105` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.422293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] RBAC: Authorizing `identity:delete_project(project_id=391132d6e00c4f04b438e886be220105)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:33.422469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 843/1680] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:33 2022] PUT /identity/v3/projects/5e7d43ccef4e4b9a9320735a98558707/users/d3fd07bd9b3b473ba7e7c40d57f44729/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:33.424594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:33.437400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.437692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.438000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.507676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] MFA Rules not processed for user `d3fd07bd9b3b473ba7e7c40d57f44729`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:33.509152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.556179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391132d6e00c4f04b438e886be220105', 'request_id': 'req-94b7cf32-f17d-460c-b0d3-884feb38380e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:33.575743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391132d6e00c4f04b438e886be220105', 'request_id': 'req-94b7cf32-f17d-460c-b0d3-884feb38380e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:33.592712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391132d6e00c4f04b438e886be220105', 'request_id': 'req-94b7cf32-f17d-460c-b0d3-884feb38380e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:33.622856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] The token cache is being invalidate because project 391132d6e00c4f04b438e886be220105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:33.623378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:33.630379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:33.637561 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-94b7cf32-f17d-460c-b0d3-884feb38380e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:33.646717 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 838/1681] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:33 2022] DELETE /identity/v3/projects/391132d6e00c4f04b438e886be220105 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:33.649389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b2ea-1b81-4cc1-bbec-801817360b33 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.652095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b2ea-1b81-4cc1-bbec-801817360b33 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.652578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0b2ea-1b81-4cc1-bbec-801817360b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.674611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 844/1682] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:33.682328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:33.684854 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.701493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4be0b2ea-1b81-4cc1-bbec-801817360b33 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:33.706911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4be0b2ea-1b81-4cc1-bbec-801817360b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.836861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024092-f2f3-4eee-b154-348bee840c5f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.844732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 839/1683] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:33.846589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cac35a-fd9a-4e06-9570-eae5fa2da315 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.846884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cac35a-fd9a-4e06-9570-eae5fa2da315 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.847166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cac35a-fd9a-4e06-9570-eae5fa2da315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.896510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-66cac35a-fd9a-4e06-9570-eae5fa2da315 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:33.897876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66cac35a-fd9a-4e06-9570-eae5fa2da315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:33.918216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 840/1684] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:33.923866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cac35a-fd9a-4e06-9570-eae5fa2da315 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:33.953740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cac35a-fd9a-4e06-9570-eae5fa2da315 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:33.958614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.958765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:33.959282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.959380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:33.959564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.959662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:33.959842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.959958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:33.960628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:33.962224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:33.964124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.011608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 841/1685] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:34.022035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:34.047585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e2ba6-2caf-419f-9536-6de5e0187268 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:34.060214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:34.060914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.061168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.061450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.062206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.064185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.120680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.131091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.153809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 842/1686] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:34 2022] POST /identity/v3/projects => generated 399 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:34.154753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:34.175814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 845/1687] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:33 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:34.227450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:34.302795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed42d20-57b6-4838-a25d-605b178e2e0a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:34.317536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:34.318261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.318545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.318841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.319523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.321781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.346074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:34.347621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:34.379033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc2103-051d-4c76-89c1-7100e8429c67 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:34.380281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 843/1688] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:34 2022] POST /identity/v3/users => generated 300 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:34.381316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:34.420187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab372ba-9d9e-46d3-92b9-9739bc43eafd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:34.441610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:34.442327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.442620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.442899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.443639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.445407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.480009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 844/1689] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:34.480793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:34.502906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c16e32-a1c5-47b7-a42b-3a88e5a8bf2c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:34.519256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:34.519504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:34.519940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.520061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.520222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.520311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.520740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.526127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.527034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.528927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:34.529133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.550978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:34.558742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 845/1690] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:34.560414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:34.584381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe90364-c225-48fa-8c1f-bef1cb5f0cd8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:34.601121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf8e13-45b1-445f-88f7-be96becc405a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:34.602277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf8e13-45b1-445f-88f7-be96becc405a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.602478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf8e13-45b1-445f-88f7-be96becc405a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.602759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf8e13-45b1-445f-88f7-be96becc405a admin admin] PATH_INFO: `/v3/projects/6a716c5bb8bc43cdb55326b425cc1b14/users/78cf60cb9c194ee5a3acaa881cda2df3/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.635914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf8e13-45b1-445f-88f7-be96becc405a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a716c5bb8bc43cdb55326b425cc1b14, user_id=78cf60cb9c194ee5a3acaa881cda2df3, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.643602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf8e13-45b1-445f-88f7-be96becc405a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.679314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:34.681083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:34.685664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 846/1691] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:34 2022] PUT /identity/v3/projects/6a716c5bb8bc43cdb55326b425cc1b14/users/78cf60cb9c194ee5a3acaa881cda2df3/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:34.687433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1be1a-f54c-464d-97a9-c0209954817e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:34.687723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1be1a-f54c-464d-97a9-c0209954817e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:34.688001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1be1a-f54c-464d-97a9-c0209954817e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:34.718871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 846/1692] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:34.780846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-79f1be1a-f54c-464d-97a9-c0209954817e None None] MFA Rules not processed for user `78cf60cb9c194ee5a3acaa881cda2df3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:34.782107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79f1be1a-f54c-464d-97a9-c0209954817e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:35.008050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 847/1693] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:35.013312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:35.030554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c405d3a-2bb6-467c-98e0-1322abd291a7 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:35.279607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa34747-9ff5-4a55-9271-8061421d77df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:35.280896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34747-9ff5-4a55-9271-8061421d77df admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.282860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34747-9ff5-4a55-9271-8061421d77df admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.287117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa34747-9ff5-4a55-9271-8061421d77df admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.290753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa34747-9ff5-4a55-9271-8061421d77df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:35.295449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa34747-9ff5-4a55-9271-8061421d77df admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:35.365891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 847/1694] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:35 2022] POST /identity/v3/projects => generated 397 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:35.372919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1be1a-f54c-464d-97a9-c0209954817e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:35.392811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1be1a-f54c-464d-97a9-c0209954817e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:35.524184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:35.536900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:35.537706 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.538139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.538427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.539321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:35.541546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:35.551746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0c58a-ff9c-46ba-8ef2-d12a491ef2b1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.552287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0c58a-ff9c-46ba-8ef2-d12a491ef2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.552754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0c58a-ff9c-46ba-8ef2-d12a491ef2b1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.562792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 848/1695] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:12:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:12:35.585004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb03d8f-1053-4361-9ab1-3af3f9dc7c33 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.585343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb03d8f-1053-4361-9ab1-3af3f9dc7c33 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.585638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb03d8f-1053-4361-9ab1-3af3f9dc7c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.619212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 848/1696] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:35 2022] POST /identity/v3/users => generated 305 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:35.624244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:35.651577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dcb03d8f-1053-4361-9ab1-3af3f9dc7c33 None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:35.655936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dcb03d8f-1053-4361-9ab1-3af3f9dc7c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:35.666983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6dd4a-c0c3-4694-b23f-356b8d583b2c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:35.701281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:35.706005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.706331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.709531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.712599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:35.723546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:35.783276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 849/1697] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:35.791014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:35.841322 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504220d6-cc5c-46e2-91b1-3a9c5b0918de admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:35.874280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:35.875266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:35.875653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:35.876036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] PATH_INFO: `/v3/projects/ab3be2d9e56b45b89dadb9ae80655981/users/c9ebe1f7f6084806ae667596e474ba87/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:35.939458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3be2d9e56b45b89dadb9ae80655981, user_id=c9ebe1f7f6084806ae667596e474ba87, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:35.942614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 849/1698] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:35.942809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:35.985167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 850/1699] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:35 2022] PUT /identity/v3/projects/ab3be2d9e56b45b89dadb9ae80655981/users/c9ebe1f7f6084806ae667596e474ba87/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:35.989404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb03d8f-1053-4361-9ab1-3af3f9dc7c33 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.015789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb03d8f-1053-4361-9ab1-3af3f9dc7c33 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.085811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:36.144218 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac7eac-4b17-40c9-8cfc-8e31fb7e8ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.144922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac7eac-4b17-40c9-8cfc-8e31fb7e8ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.145212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac7eac-4b17-40c9-8cfc-8e31fb7e8ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.145495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac7eac-4b17-40c9-8cfc-8e31fb7e8ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.147221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac7eac-4b17-40c9-8cfc-8e31fb7e8ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.149123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ac7eac-4b17-40c9-8cfc-8e31fb7e8ffb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.177591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 850/1700] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:36.182226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.202142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906bf58-f123-4a98-b7ba-bab93b2e5b6d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.216537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803a8b0-29d4-4800-9eaf-fffe505d7d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.218034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a8b0-29d4-4800-9eaf-fffe505d7d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.219911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a8b0-29d4-4800-9eaf-fffe505d7d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.220229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a8b0-29d4-4800-9eaf-fffe505d7d7b admin admin] PATH_INFO: `/v3/projects/ab3be2d9e56b45b89dadb9ae80655981/users/c9ebe1f7f6084806ae667596e474ba87/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.227434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803a8b0-29d4-4800-9eaf-fffe505d7d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab3be2d9e56b45b89dadb9ae80655981, user_id=c9ebe1f7f6084806ae667596e474ba87, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.236802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803a8b0-29d4-4800-9eaf-fffe505d7d7b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.267846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 851/1701] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:36 2022] PUT /identity/v3/projects/ab3be2d9e56b45b89dadb9ae80655981/users/c9ebe1f7f6084806ae667596e474ba87/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:36.279078 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6eed-3023-41ab-97ac-0d51aaa30087 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.279318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6eed-3023-41ab-97ac-0d51aaa30087 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.279587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f6eed-3023-41ab-97ac-0d51aaa30087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.367925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-544f6eed-3023-41ab-97ac-0d51aaa30087 None None] MFA Rules not processed for user `c9ebe1f7f6084806ae667596e474ba87`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:36.372025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-544f6eed-3023-41ab-97ac-0d51aaa30087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:36.520880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffe455-3935-4b9c-bb2b-ab5c31072490 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.521171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffe455-3935-4b9c-bb2b-ab5c31072490 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.522046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffe455-3935-4b9c-bb2b-ab5c31072490 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.525150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 852/1702] 10.222.0.88 () {54 vars in 960 bytes} [Thu Jun 16 16:12:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:12:36.538559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a0df0-2744-40a4-be26-12cb4d0d58f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.539055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a0df0-2744-40a4-be26-12cb4d0d58f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.539488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a0df0-2744-40a4-be26-12cb4d0d58f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.557410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 851/1703] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:36.571487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f6eed-3023-41ab-97ac-0d51aaa30087 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.593387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-275a0df0-2744-40a4-be26-12cb4d0d58f2 None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:36.598403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-275a0df0-2744-40a4-be26-12cb4d0d58f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:36.767441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f6eed-3023-41ab-97ac-0d51aaa30087 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.785050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.786011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.786503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.787626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.809388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.821125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.835169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 853/1704] 10.222.0.88 () {58 vars in 1057 bytes} [Thu Jun 16 16:12:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:36.848244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a0df0-2744-40a4-be26-12cb4d0d58f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:36.860864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 852/1705] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:36.873957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a0df0-2744-40a4-be26-12cb4d0d58f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:36.899351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca41c78-2d6c-4245-8073-1aa84b314f84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:36.900034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca41c78-2d6c-4245-8073-1aa84b314f84 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:36.900370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca41c78-2d6c-4245-8073-1aa84b314f84 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:36.900605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca41c78-2d6c-4245-8073-1aa84b314f84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:36.930678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca41c78-2d6c-4245-8073-1aa84b314f84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:36.934040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca41c78-2d6c-4245-8073-1aa84b314f84 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:36.989910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 854/1706] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:37.203870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:37.214356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b6c06-89d4-47c7-a720-15accdcb170b service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:37.218972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:37.219707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.220065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.220754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.344190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:37.350242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:37.352749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:37.379828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 853/1707] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:37.778870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809794df-bd94-4a58-a9f6-d9b77546dac3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:37.779171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809794df-bd94-4a58-a9f6-d9b77546dac3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:37.779449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809794df-bd94-4a58-a9f6-d9b77546dac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:37.875617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-809794df-bd94-4a58-a9f6-d9b77546dac3 None None] MFA Rules not processed for user `46445560ea7e496fa6c9c2e7c9a5b747`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:37.885908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-809794df-bd94-4a58-a9f6-d9b77546dac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:38.104826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 855/1708] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:38.119381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:38.120805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:38.540006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469d0a66-c3fd-4b3b-86bd-9dec56875d6d service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:38.554618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a391b-1552-4a37-9dab-bdb40dee810e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:38.557478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a391b-1552-4a37-9dab-bdb40dee810e service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:38.560721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a391b-1552-4a37-9dab-bdb40dee810e service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:38.562398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a391b-1552-4a37-9dab-bdb40dee810e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:38.586504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a391b-1552-4a37-9dab-bdb40dee810e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:38.588417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a391b-1552-4a37-9dab-bdb40dee810e service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:38.619088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 854/1709] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:39.509709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:39.510700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:39.511190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:39.560857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:39.580045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] MFA Rules not processed for user `5e2021bba1874f0fb4e6904aac576431`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:39.586264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:39.843309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 856/1710] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:39.847931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122986e-6ea6-4e90-8e5f-a98cb6e4e82a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:39.848920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122986e-6ea6-4e90-8e5f-a98cb6e4e82a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:39.849416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122986e-6ea6-4e90-8e5f-a98cb6e4e82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:39.908576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c122986e-6ea6-4e90-8e5f-a98cb6e4e82a None None] MFA Rules not processed for user `5e2021bba1874f0fb4e6904aac576431`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:39.919318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c122986e-6ea6-4e90-8e5f-a98cb6e4e82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:39.955983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 855/1711] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:39.967505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:39.969399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:40.007664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb846f78-0205-4f3f-957f-fcdacbfa8def None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:40.041269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f223caf-6033-44a3-bdff-baff6264c9f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:40.042004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223caf-6033-44a3-bdff-baff6264c9f9 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:40.042292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223caf-6033-44a3-bdff-baff6264c9f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:40.042574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f223caf-6033-44a3-bdff-baff6264c9f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:40.072035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f223caf-6033-44a3-bdff-baff6264c9f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:40.077880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f223caf-6033-44a3-bdff-baff6264c9f9 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:40.108438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 857/1712] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:40.320160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a884db-a2b5-4edb-89ae-d2c5ce5487ef None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:40.320450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a884db-a2b5-4edb-89ae-d2c5ce5487ef None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:40.320734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a884db-a2b5-4edb-89ae-d2c5ce5487ef None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:40.324555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 856/1713] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:12:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:40.362876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:40.363174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:40.363454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:40.418228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] MFA Rules not processed for user `7512a42da9154c008dad057c5508a02b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:40.420479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:40.453249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 858/1714] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:12:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:40.610691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a884db-a2b5-4edb-89ae-d2c5ce5487ef None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:40.636761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a884db-a2b5-4edb-89ae-d2c5ce5487ef None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:40.660507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a0e01-29d1-4b76-a0b7-8baa9a6df631 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:40.663350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a0e01-29d1-4b76-a0b7-8baa9a6df631 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:40.663677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a0e01-29d1-4b76-a0b7-8baa9a6df631 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:40.663974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a0e01-29d1-4b76-a0b7-8baa9a6df631 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:40.700872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a0e01-29d1-4b76-a0b7-8baa9a6df631 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:40.702579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a0e01-29d1-4b76-a0b7-8baa9a6df631 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:40.725541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:40.728295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:40.764913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 857/1715] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:40.923125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e73f5a-7254-4b92-a61e-376496a51c0f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:40.935234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:40.936888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:40.937445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:40.938090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] PATH_INFO: `/v3/users/5fcb3530bbd84c219c76e1f3005546a8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:40.939937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fcb3530bbd84c219c76e1f3005546a8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:40.943388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:41.068732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fcb3530bbd84c219c76e1f3005546a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:41.069581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fcb3530bbd84c219c76e1f3005546a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:41.070318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fcb3530bbd84c219c76e1f3005546a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:41.079378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fcb3530bbd84c219c76e1f3005546a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:41.089007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a3ebbd4-f120-4fcb-9711-9e6b740c1c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fcb3530bbd84c219c76e1f3005546a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:41.124276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:41.133855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 859/1716] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:40 2022] DELETE /identity/v3/users/5fcb3530bbd84c219c76e1f3005546a8 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:41.258309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:41.258841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:41.259153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:41.355111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] MFA Rules not processed for user `16b8389777744e6581ad14be85731f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:41.359284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:41.607997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9de412-b7ce-4b08-8729-769201d2c5b5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:41.608505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9de412-b7ce-4b08-8729-769201d2c5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:41.608927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9de412-b7ce-4b08-8729-769201d2c5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:41.620098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 858/1717] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:41.643184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4f9de412-b7ce-4b08-8729-769201d2c5b5 None None] MFA Rules not processed for user `16b8389777744e6581ad14be85731f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:41.645570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f9de412-b7ce-4b08-8729-769201d2c5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:41.658414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 860/1718] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:41.672068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:41.674158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:41.781508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:41.850042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af849d-d864-4b46-997f-6df2cf26e4f7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:41.864504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:41.865345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:41.865785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:41.866252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:41.878320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:41.880373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:41.894888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 859/1719] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:42.030546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636739a-721d-4042-b376-56730629aa9e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.031041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636739a-721d-4042-b376-56730629aa9e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.031153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636739a-721d-4042-b376-56730629aa9e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.035547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 861/1720] 10.222.0.88 () {54 vars in 912 bytes} [Thu Jun 16 16:12:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:42.037791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:42.039666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:42.044150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.045886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.046183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.175041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] MFA Rules not processed for user `33e61ed31e0943e8baf9a85412860245`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:42.176325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:42.373391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0cb95-f822-4118-9e72-023f2fcf83e9 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:42.375352 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 862/1721] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:42.389420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:42.390359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.390646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.391018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] PATH_INFO: `/v3/users/8e7fbb5511824b549c3ef1e306138ade` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.394501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7fbb5511824b549c3ef1e306138ade)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:42.396496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:42.446104 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:42.447862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:42.468633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573438b8-f90f-422b-8039-7def1cd10eda None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:42.490749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:42.491723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.492130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.492418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.519865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:42.521399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:42.536471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 863/1722] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:42.582545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7fbb5511824b549c3ef1e306138ade'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.585959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7fbb5511824b549c3ef1e306138ade'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.599385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:42.601098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:42.615237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7fbb5511824b549c3ef1e306138ade'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.620366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7fbb5511824b549c3ef1e306138ade'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.647981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bf7045cc-96db-42e1-b7d5-6d37a984afa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7fbb5511824b549c3ef1e306138ade'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.654461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8766fa-895d-4d06-b21a-be851ba0917e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:42.670371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:42.672208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.672953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.673689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] PATH_INFO: `/v3/projects/125583532148403bad4a146c2d3be672` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.680624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=125583532148403bad4a146c2d3be672)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:42.686874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:42.718625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 860/1723] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:42 2022] DELETE /identity/v3/users/8e7fbb5511824b549c3ef1e306138ade => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:42.723862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:42.724590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:42.724892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:42.753875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:42.759285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:42.874487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '125583532148403bad4a146c2d3be672', 'request_id': 'req-7d8b298a-a2e8-4a4f-af73-8f1163d31528'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.898642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '125583532148403bad4a146c2d3be672', 'request_id': 'req-7d8b298a-a2e8-4a4f-af73-8f1163d31528'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.909541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:42.913400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '125583532148403bad4a146c2d3be672', 'request_id': 'req-7d8b298a-a2e8-4a4f-af73-8f1163d31528'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.943444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] The token cache is being invalidate because project 125583532148403bad4a146c2d3be672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:42.943820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.946117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:42.946744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 861/1724] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:12:42.951091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 864/1725] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:42 2022] DELETE /identity/v3/projects/125583532148403bad4a146c2d3be672 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:42.955345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:42.958012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:42.997857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2858-5d5f-483a-83d6-79839a54077a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.058146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.059724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:43.180761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.181677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.182007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.182289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] PATH_INFO: `/v3/users/7dd332d52c9b47fe9cf9ffa1b33722e1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.186143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd332d52c9b47fe9cf9ffa1b33722e1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.188120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.215219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b298a-a2e8-4a4f-af73-8f1163d31528 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.251018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87ebc7-5906-43a4-90c1-0e82aeb1d357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.253041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87ebc7-5906-43a4-90c1-0e82aeb1d357 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.253354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87ebc7-5906-43a4-90c1-0e82aeb1d357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.253891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87ebc7-5906-43a4-90c1-0e82aeb1d357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.257271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c87ebc7-5906-43a4-90c1-0e82aeb1d357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:43.326853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87ebc7-5906-43a4-90c1-0e82aeb1d357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.329367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87ebc7-5906-43a4-90c1-0e82aeb1d357 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.349091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd332d52c9b47fe9cf9ffa1b33722e1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.350279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd332d52c9b47fe9cf9ffa1b33722e1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.368210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd332d52c9b47fe9cf9ffa1b33722e1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.370390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 865/1726] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:12:43.371115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd332d52c9b47fe9cf9ffa1b33722e1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.372525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24536480-6644-47a5-9c0e-977413afb9e0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.372820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24536480-6644-47a5-9c0e-977413afb9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.373105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24536480-6644-47a5-9c0e-977413afb9e0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.380198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 866/1727] 10.222.0.88 () {54 vars in 912 bytes} [Thu Jun 16 16:12:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:43.387731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.388059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.388309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.391062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd332d52c9b47fe9cf9ffa1b33722e1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:43.398068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:43.439718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] MFA Rules not processed for user `33e61ed31e0943e8baf9a85412860245`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:43.440757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:43.467484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 862/1728] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:42 2022] DELETE /identity/v3/users/7dd332d52c9b47fe9cf9ffa1b33722e1 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:43.476912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 867/1729] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:43.673851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:43.677374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:43.856994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190bc1af-3322-4e9b-8f5d-5b8b61604519 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:43.861087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:43.861883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:43.862075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:43.862362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:43.888410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:43.898344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:43.922320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 863/1730] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:44.322830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.325548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:44.555070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce0f61-b6c3-47df-8f76-d2a20b448165 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.591945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.594009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:44.633051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:44.751740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcfba7-7e60-4d0a-aa59-f95d3c5cc18a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.755460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcfba7-7e60-4d0a-aa59-f95d3c5cc18a service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.755860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcfba7-7e60-4d0a-aa59-f95d3c5cc18a service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.756303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcfba7-7e60-4d0a-aa59-f95d3c5cc18a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.759786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0fcfba7-7e60-4d0a-aa59-f95d3c5cc18a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:44.767784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8c6d6-5557-4a2a-8443-262eb4bcec2c service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:44.776257 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:44.776463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcfba7-7e60-4d0a-aa59-f95d3c5cc18a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.777227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.777516 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.777829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] PATH_INFO: `/v3/users/78cf60cb9c194ee5a3acaa881cda2df3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.778360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcfba7-7e60-4d0a-aa59-f95d3c5cc18a service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.785351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78cf60cb9c194ee5a3acaa881cda2df3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:44.791358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:44.810979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 868/1731] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:44.815362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20d3d8-086a-49b6-a8d7-6aea8ce24f79 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:44.816579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20d3d8-086a-49b6-a8d7-6aea8ce24f79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:44.817681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20d3d8-086a-49b6-a8d7-6aea8ce24f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:44.900655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78cf60cb9c194ee5a3acaa881cda2df3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:44.902553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78cf60cb9c194ee5a3acaa881cda2df3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:44.918658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5b20d3d8-086a-49b6-a8d7-6aea8ce24f79 None None] MFA Rules not processed for user `bfbcc9b45bc341228bcc0f2192040224`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:44.921538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78cf60cb9c194ee5a3acaa881cda2df3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:44.922270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5b20d3d8-086a-49b6-a8d7-6aea8ce24f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:44.924038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78cf60cb9c194ee5a3acaa881cda2df3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:44.946326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78cf60cb9c194ee5a3acaa881cda2df3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:44.985032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 864/1732] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:44 2022] DELETE /identity/v3/users/78cf60cb9c194ee5a3acaa881cda2df3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:44.986980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:44.995379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:45.145110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 869/1733] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:45.146986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:45.147381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:45.147961 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:45.179470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:45.182116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:45.250957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464a496-c704-4663-bd6e-2314a2e471a7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:45.257449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 870/1734] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:45.261452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:45.265553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:45.283661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:45.284605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:45.284949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:45.285154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] PATH_INFO: `/v3/users/b2757df3207a404a834babbf189fdf95` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:45.292929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2757df3207a404a834babbf189fdf95)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:45.296004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:45.328212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:45.433771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:45.474114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2757df3207a404a834babbf189fdf95'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:45.474894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2757df3207a404a834babbf189fdf95'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:45.491926 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2757df3207a404a834babbf189fdf95'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:45.493297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2757df3207a404a834babbf189fdf95'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:45.509368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2757df3207a404a834babbf189fdf95'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:45.561065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87e24a-353d-4f92-b0e4-0dd012b9aa21 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:45.572782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 865/1735] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:44 2022] DELETE /identity/v3/users/b2757df3207a404a834babbf189fdf95 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:45.573518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:45.579569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:45.598457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6b890-b4e5-4e94-97fc-2e3836bb5d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:45.599828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6b890-b4e5-4e94-97fc-2e3836bb5d41 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:45.600274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6b890-b4e5-4e94-97fc-2e3836bb5d41 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:45.600747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6b890-b4e5-4e94-97fc-2e3836bb5d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:45.603286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5e6b890-b4e5-4e94-97fc-2e3836bb5d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:45.779262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6b890-b4e5-4e94-97fc-2e3836bb5d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:45.794966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6b890-b4e5-4e94-97fc-2e3836bb5d41 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:45.800287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577a607-4574-4d47-8f16-b54d18c020ef admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:45.819200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 871/1736] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:45.820545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:45.821994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:45.822419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:45.822701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:45.823713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:45.825702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:45.826842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] PATH_INFO: `/v3/projects/bfa27850e0d247a982554a2065e54f43` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:45.832367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa27850e0d247a982554a2065e54f43)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:45.833912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 16:12:45.833912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: warnings.warn(msg) Jun 16 16:12:45.833912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: Jun 16 16:12:45.835949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:45.863986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] MFA Rules not processed for user `46445560ea7e496fa6c9c2e7c9a5b747`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:45.866967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:46.057530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa27850e0d247a982554a2065e54f43', 'request_id': 'req-4d310491-b501-48c4-8b6c-aec10db196bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.106386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa27850e0d247a982554a2065e54f43', 'request_id': 'req-4d310491-b501-48c4-8b6c-aec10db196bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.173911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 872/1737] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:46.174821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa27850e0d247a982554a2065e54f43', 'request_id': 'req-4d310491-b501-48c4-8b6c-aec10db196bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.175183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:46.186203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:46.223812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] The token cache is being invalidate because project bfa27850e0d247a982554a2065e54f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:46.226064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.229810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.258082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 866/1738] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:45 2022] DELETE /identity/v3/projects/bfa27850e0d247a982554a2065e54f43 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:46.258082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:46.263864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:46.334673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fb995-dfc9-4f89-8836-0ef7bac4e8a4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:46.359244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:46.440118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:46.471757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b539a2-9e0f-4b43-a432-68d1dcff3550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:46.474634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b539a2-9e0f-4b43-a432-68d1dcff3550 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:46.475218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b539a2-9e0f-4b43-a432-68d1dcff3550 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:46.475660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b539a2-9e0f-4b43-a432-68d1dcff3550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:46.479703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90b539a2-9e0f-4b43-a432-68d1dcff3550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:46.492497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d310491-b501-48c4-8b6c-aec10db196bb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:46.512897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b539a2-9e0f-4b43-a432-68d1dcff3550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:46.514640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b539a2-9e0f-4b43-a432-68d1dcff3550 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:46.522968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:46.524300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:46.524804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:46.525207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] PATH_INFO: `/v3/projects/5df9f0e03bc94dc9916183aee67492bd` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:46.528262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df9f0e03bc94dc9916183aee67492bd)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:46.535737 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:46.541413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 873/1739] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:46.544882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06ba66-8677-4371-bd22-10a4cb7d6eaf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:46.544882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06ba66-8677-4371-bd22-10a4cb7d6eaf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:46.544882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06ba66-8677-4371-bd22-10a4cb7d6eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:46.593008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a06ba66-8677-4371-bd22-10a4cb7d6eaf None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:46.594460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a06ba66-8677-4371-bd22-10a4cb7d6eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:46.623348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 874/1740] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:46.623348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:46.623348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:46.623348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:46.666087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] MFA Rules not processed for user `16b8389777744e6581ad14be85731f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:46.671024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:46.699128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df9f0e03bc94dc9916183aee67492bd', 'request_id': 'req-cf2cf54f-404c-4f70-a155-07ac130b98e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.713780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df9f0e03bc94dc9916183aee67492bd', 'request_id': 'req-cf2cf54f-404c-4f70-a155-07ac130b98e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.731011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df9f0e03bc94dc9916183aee67492bd', 'request_id': 'req-cf2cf54f-404c-4f70-a155-07ac130b98e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.763794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] The token cache is being invalidate because project 5df9f0e03bc94dc9916183aee67492bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:46.765492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.770511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cf2cf54f-404c-4f70-a155-07ac130b98e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:46.777166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 867/1741] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:46 2022] DELETE /identity/v3/projects/5df9f0e03bc94dc9916183aee67492bd => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:46.779250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:46.779822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:46.780316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:46.841250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:46.847269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 875/1742] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:46.847851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:46.848208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:46.870279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:47.042871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed68c04-a41c-4421-aca5-a089c78cb730 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:47.051220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8945fc-0bea-4a2b-8833-d2c723420e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:47.052282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8945fc-0bea-4a2b-8833-d2c723420e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:47.052590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8945fc-0bea-4a2b-8833-d2c723420e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:47.055956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8945fc-0bea-4a2b-8833-d2c723420e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:47.070840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8c8945fc-0bea-4a2b-8833-d2c723420e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:47.147273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 868/1743] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:47.149753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:47.157295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:47.219860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be2f3-1208-4bb5-b3e0-f1cf88f8e34d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:47.231978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:47.233667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:47.235055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:47.235188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] PATH_INFO: `/v3/users/d3fd07bd9b3b473ba7e7c40d57f44729` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:47.238508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3fd07bd9b3b473ba7e7c40d57f44729)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:47.243314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:47.250970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8945fc-0bea-4a2b-8833-d2c723420e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:47.253477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8945fc-0bea-4a2b-8833-d2c723420e97 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:47.267570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:47.290936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 876/1744] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:47.300933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55817aa-39bd-433e-b532-156b2ccedfcc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:47.301371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55817aa-39bd-433e-b532-156b2ccedfcc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:47.301656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55817aa-39bd-433e-b532-156b2ccedfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:47.401596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3fd07bd9b3b473ba7e7c40d57f44729'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:47.402861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3fd07bd9b3b473ba7e7c40d57f44729'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:47.405578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c55817aa-39bd-433e-b532-156b2ccedfcc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:47.413035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c55817aa-39bd-433e-b532-156b2ccedfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:47.430243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3fd07bd9b3b473ba7e7c40d57f44729'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:47.431054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3fd07bd9b3b473ba7e7c40d57f44729'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:47.449369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 877/1745] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:47.452898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55817aa-39bd-433e-b532-156b2ccedfcc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:47.454041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3fd07bd9b3b473ba7e7c40d57f44729'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:47.477765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:47.481505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55817aa-39bd-433e-b532-156b2ccedfcc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:47.495617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:47.496889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 869/1746] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:47 2022] DELETE /identity/v3/users/d3fd07bd9b3b473ba7e7c40d57f44729 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:47.497015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:47.497482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:47.498072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:47.499093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:47.499414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:47.511514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:47.540346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:47.542333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 878/1747] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:47.679916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6786fff-b6bf-4cab-a170-b25315bac4d6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:47.687118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:47.688337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:47.688822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:47.689242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:47.690234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:47.693203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:47.735440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 870/1748] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:47 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:47.748704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:47.776730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acbc7a-f06e-4570-a241-cff393d668ac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:47.973096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:47.973971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:47.974120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:47.974288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:47.975054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:47.977185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:48.026555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 879/1749] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:47 2022] POST /identity/v3/users => generated 287 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:48.031579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:48.055710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c780f53-016e-4c67-8dbb-2cdbd5707883 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:48.066392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:48.068525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:48.070093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:48.072790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:48.074013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:48.078795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:48.081414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:48.083389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:48.106689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 871/1750] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:48.108533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e0931-a76c-4c04-9a52-54e5ede90361 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:48.112280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:48.120598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:48.121548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:48.121869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:48.122156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] PATH_INFO: `/v3/projects/6a716c5bb8bc43cdb55326b425cc1b14` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:48.123803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a716c5bb8bc43cdb55326b425cc1b14)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:48.126441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:48.149130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088252-55bb-48fb-8d9a-090e591df250 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:48.168953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:48.170190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:48.170699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:48.171210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:48.172153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:48.183443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:48.227716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 872/1751] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:48.235546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:48.269835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7964677-a132-4670-bdbd-b5cf8cf8c27e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:48.289773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a716c5bb8bc43cdb55326b425cc1b14', 'request_id': 'req-4d10c963-4d25-4a51-894c-440db079875a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.315231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a716c5bb8bc43cdb55326b425cc1b14', 'request_id': 'req-4d10c963-4d25-4a51-894c-440db079875a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.345469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a716c5bb8bc43cdb55326b425cc1b14', 'request_id': 'req-4d10c963-4d25-4a51-894c-440db079875a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.391045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] The token cache is being invalidate because project 6a716c5bb8bc43cdb55326b425cc1b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:48.391344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.398036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.407444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 880/1752] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:48 2022] DELETE /identity/v3/projects/6a716c5bb8bc43cdb55326b425cc1b14 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:48.410515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:48.419549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:48.421780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:48.425322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:48.427134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:48.427572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] PATH_INFO: `/v3/projects/afb4aa091d4c4f9aa666b72519cc9565/users/414bb0cab9194ee4a098461d1a1fbb13/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:48.466999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10c963-4d25-4a51-894c-440db079875a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:48.481669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb4aa091d4c4f9aa666b72519cc9565, user_id=414bb0cab9194ee4a098461d1a1fbb13, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:48.489490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:48.490400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:48.490683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:48.490949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] PATH_INFO: `/v3/projects/2dd88144811c46e3889b66c90df5f38f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:48.495685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd88144811c46e3889b66c90df5f38f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:48.498249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:48.515311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:48.559610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 873/1753] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:48 2022] PUT /identity/v3/projects/afb4aa091d4c4f9aa666b72519cc9565/users/414bb0cab9194ee4a098461d1a1fbb13/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:48.561941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:48.564707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:48.685330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:48.697529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd88144811c46e3889b66c90df5f38f', 'request_id': 'req-674d8811-e598-418f-a499-0ce0a9760086'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.719168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd88144811c46e3889b66c90df5f38f', 'request_id': 'req-674d8811-e598-418f-a499-0ce0a9760086'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.741632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd88144811c46e3889b66c90df5f38f', 'request_id': 'req-674d8811-e598-418f-a499-0ce0a9760086'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.781849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:48.783518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] The token cache is being invalidate because project 2dd88144811c46e3889b66c90df5f38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:48.783961 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.785938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-674d8811-e598-418f-a499-0ce0a9760086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:48.790914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 881/1754] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:48 2022] DELETE /identity/v3/projects/2dd88144811c46e3889b66c90df5f38f => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:48.793421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19eefaf-8269-47e5-81ed-c479d11bc865 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:48.796058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19eefaf-8269-47e5-81ed-c479d11bc865 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:48.796471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19eefaf-8269-47e5-81ed-c479d11bc865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:48.867450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c19eefaf-8269-47e5-81ed-c479d11bc865 None None] MFA Rules not processed for user `414bb0cab9194ee4a098461d1a1fbb13`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:48.869121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c19eefaf-8269-47e5-81ed-c479d11bc865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:48.881479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ceec5-483a-49dc-9952-7fc1e3a524c4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:49.094376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:49.094904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 882/1755] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:49.105477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:49.106344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:49.106784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:49.107200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] PATH_INFO: `/v3/users/c9ebe1f7f6084806ae667596e474ba87` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:49.108437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:49.122776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:49.124387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ebe1f7f6084806ae667596e474ba87)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:49.134485 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:49.185741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:49.195122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:49.283913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ebe1f7f6084806ae667596e474ba87'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.285225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ebe1f7f6084806ae667596e474ba87'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.302232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ebe1f7f6084806ae667596e474ba87'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.303052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ebe1f7f6084806ae667596e474ba87'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.318947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ebe1f7f6084806ae667596e474ba87'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.374217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 883/1756] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:12:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:49.374388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 874/1757] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:48 2022] DELETE /identity/v3/users/c9ebe1f7f6084806ae667596e474ba87 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:49.375585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:49.377597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:49.378223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:49.381389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:49.410673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fedc43-aba7-4b91-8284-5d00fe64e0c0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:49.563612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79750f-6d5b-4f48-9796-ab21192629e6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:49.572150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:49.573599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:49.573977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:49.574668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] PATH_INFO: `/v3/users/f8781ce43cb54045b5929d94faf74b51` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:49.577249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8781ce43cb54045b5929d94faf74b51)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:49.578209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:49.579390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:49.580730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:49.582633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:49.582962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] PATH_INFO: `/v3/projects/5e7d43ccef4e4b9a9320735a98558707` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:49.587411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7d43ccef4e4b9a9320735a98558707)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:49.611147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:49.617080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:49.623860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:49.736159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8781ce43cb54045b5929d94faf74b51'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.738116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8781ce43cb54045b5929d94faf74b51'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.739293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8781ce43cb54045b5929d94faf74b51'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.761592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8781ce43cb54045b5929d94faf74b51'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.771075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7d43ccef4e4b9a9320735a98558707', 'request_id': 'req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.781575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8781ce43cb54045b5929d94faf74b51'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.788590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7d43ccef4e4b9a9320735a98558707', 'request_id': 'req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.812103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7d43ccef4e4b9a9320735a98558707', 'request_id': 'req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.833274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 884/1758] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:49 2022] DELETE /identity/v3/users/f8781ce43cb54045b5929d94faf74b51 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:49.835292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:49.835998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:49.858849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] The token cache is being invalidate because project 5e7d43ccef4e4b9a9320735a98558707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:49.859146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.861048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:49.866234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 875/1759] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:49 2022] DELETE /identity/v3/projects/5e7d43ccef4e4b9a9320735a98558707 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:49.870400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:49.872317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:50.024373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86fd01-7829-48f3-ab94-9545e9e3ce50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.057063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e2caa-8748-48fe-9fe4-1e396d213377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.060088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2caa-8748-48fe-9fe4-1e396d213377 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.065394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2caa-8748-48fe-9fe4-1e396d213377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.067867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2caa-8748-48fe-9fe4-1e396d213377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.071612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e2caa-8748-48fe-9fe4-1e396d213377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.072796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec0deb-8f01-41e7-a7ff-6f59d46236f9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.078710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e2caa-8748-48fe-9fe4-1e396d213377 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.175360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 885/1760] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:49 2022] POST /identity/v3/projects => generated 373 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:50.182348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.187164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.189586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.279410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f154089-67cc-4a8d-98dc-74c50811614c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.280654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f154089-67cc-4a8d-98dc-74c50811614c service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.282264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f154089-67cc-4a8d-98dc-74c50811614c service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.282835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f154089-67cc-4a8d-98dc-74c50811614c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.284461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:50.289735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3f154089-67cc-4a8d-98dc-74c50811614c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:50.290682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:50.427562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:50.551711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f154089-67cc-4a8d-98dc-74c50811614c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.555244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f154089-67cc-4a8d-98dc-74c50811614c service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.584967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 886/1761] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:50.586935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 876/1762] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:12:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:12:50.587057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:50.589810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:50.592934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56df64-6ba1-47b3-a220-6f96f153d223 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.593494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56df64-6ba1-47b3-a220-6f96f153d223 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.593942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56df64-6ba1-47b3-a220-6f96f153d223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.627639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bffd9-3269-413b-8b14-131fff110e29 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.637945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.639757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.640261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.641153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.642000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5c56df64-6ba1-47b3-a220-6f96f153d223 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:50.642882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.646113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5c56df64-6ba1-47b3-a220-6f96f153d223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:50.647993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.669938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 877/1763] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:50.674155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56df64-6ba1-47b3-a220-6f96f153d223 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:50.686127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:50.690798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56df64-6ba1-47b3-a220-6f96f153d223 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.714424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 887/1764] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:50 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:50.719353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:50.743066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.744200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.744730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.745371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.746748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:50.747281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.748628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cdf7-bb04-4ac5-ad0d-4b5c897c5aa8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.749974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.765885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.767734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.768272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.768985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.770265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.772586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.797279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 878/1765] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:50.799433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:50.805954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 888/1766] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:50.807708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:50.821579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887593-dc18-4f58-ad88-906da18b103c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.831955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.832773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.833315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.833674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.834608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.835553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac6e60-184b-48fc-b177-434a5c07eb9d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:50.837397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:50.862159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:50.867166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:50.867939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:50.869908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] PATH_INFO: `/v3/projects/d3a03a4d3617423eb0dad1f31db24ce7/users/c6d722263c684655afa7d020faa015c7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:50.900177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 879/1767] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:50 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:50.914120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:50.943685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a03a4d3617423eb0dad1f31db24ce7, user_id=c6d722263c684655afa7d020faa015c7, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:50.951308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:51.000748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 889/1768] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:50 2022] PUT /identity/v3/projects/d3a03a4d3617423eb0dad1f31db24ce7/users/c6d722263c684655afa7d020faa015c7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:51.006257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:51.268591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24506eac-8ad7-415f-842f-f8d23019d2e3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:51.373562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1442-a75d-4234-b5e1-7dfdfcf2fbf2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:51.400989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:51.401683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.402004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.402317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.403075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:51.417459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:51.458359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 890/1769] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:51.460118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:51.523704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d85423-77c0-441e-b395-9a47220100dd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:51.531985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336eb151-118c-457c-9e5f-bdac71b6d0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:51.534988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb151-118c-457c-9e5f-bdac71b6d0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.535522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb151-118c-457c-9e5f-bdac71b6d0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.538378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336eb151-118c-457c-9e5f-bdac71b6d0ae admin admin] PATH_INFO: `/v3/projects/d3a03a4d3617423eb0dad1f31db24ce7/users/c6d722263c684655afa7d020faa015c7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.540999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eb151-118c-457c-9e5f-bdac71b6d0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a03a4d3617423eb0dad1f31db24ce7, user_id=c6d722263c684655afa7d020faa015c7, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:51.546137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336eb151-118c-457c-9e5f-bdac71b6d0ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:51.594072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:51.596034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.615791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 891/1770] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:51 2022] PUT /identity/v3/projects/d3a03a4d3617423eb0dad1f31db24ce7/users/c6d722263c684655afa7d020faa015c7/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:51.627210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.627210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.627210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:51.627210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:51.631028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:51.632746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:51.633057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:51.719863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] MFA Rules not processed for user `c6d722263c684655afa7d020faa015c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:51.721367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:51.739677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 880/1771] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:50 2022] POST /identity/v3/users => generated 292 bytes in 835 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:51.747606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:51.782214 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780039-b09e-40da-97f1-c0197d4a5a8e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:51.848364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:51.990821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:52.006469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.006934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.007560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.008506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:52.010536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:52.013658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 892/1772] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:52.014607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.027585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.075648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d2d00-644b-40f6-8ec4-82eb5fb2cdcd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:52.086152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:52.089468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.093847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 881/1773] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:52.093847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.094087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.094087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] PATH_INFO: `/v3/projects/ab3be2d9e56b45b89dadb9ae80655981` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.099203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab3be2d9e56b45b89dadb9ae80655981)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:52.106644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:52.110928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.268088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:52.306907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933b5e2-feae-4680-a1ae-f27c401ae9c5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:52.343143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab3be2d9e56b45b89dadb9ae80655981', 'request_id': 'req-e04528e5-dd27-479c-84b4-be202f404917'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.359983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab3be2d9e56b45b89dadb9ae80655981', 'request_id': 'req-e04528e5-dd27-479c-84b4-be202f404917'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.369114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab3be2d9e56b45b89dadb9ae80655981', 'request_id': 'req-e04528e5-dd27-479c-84b4-be202f404917'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.409968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] The token cache is being invalidate because project ab3be2d9e56b45b89dadb9ae80655981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:52.410652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.416176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:52.425604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 893/1774] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:52 2022] DELETE /identity/v3/projects/ab3be2d9e56b45b89dadb9ae80655981 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:52.425604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.426535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.498595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:52.499293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.499587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.499874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.506933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.652929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04528e5-dd27-479c-84b4-be202f404917 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:52.671003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1450e5-7d38-45f1-ad83-04b8999bf010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:52.671690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1450e5-7d38-45f1-ad83-04b8999bf010 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.671977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1450e5-7d38-45f1-ad83-04b8999bf010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.672259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1450e5-7d38-45f1-ad83-04b8999bf010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.682648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1450e5-7d38-45f1-ad83-04b8999bf010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:52.686176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1450e5-7d38-45f1-ad83-04b8999bf010 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:52.754942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 894/1775] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:52.770651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10457219-baa7-49b3-b944-25c58cc7ab7a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.772693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10457219-baa7-49b3-b944-25c58cc7ab7a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.773068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10457219-baa7-49b3-b944-25c58cc7ab7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.827759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:52.833002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:52.867801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10457219-baa7-49b3-b944-25c58cc7ab7a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:52.869299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10457219-baa7-49b3-b944-25c58cc7ab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.884211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 882/1776] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 793 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:52.886538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:52.906573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 895/1777] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:52.910551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c816-3c28-4979-84de-ca6f576394ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:52.910845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c816-3c28-4979-84de-ca6f576394ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:52.911388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c816-3c28-4979-84de-ca6f576394ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:52.950924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee94280-3a58-4ccd-a2d3-094b77fc9dc6 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:52.988759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6495c816-3c28-4979-84de-ca6f576394ca None None] MFA Rules not processed for user `33e61ed31e0943e8baf9a85412860245`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:52.992321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6495c816-3c28-4979-84de-ca6f576394ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:52.993122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.000228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.002781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.003229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] PATH_INFO: `/v3/projects/8fcc970c5fe342ff9e9abd75df5be88d/users/c7ed83b38be247899991e25cb302524f/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.086797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcc970c5fe342ff9e9abd75df5be88d, user_id=c7ed83b38be247899991e25cb302524f, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.094050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.138264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 896/1778] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jun 16 16:12:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:53.140394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0bee6-08dd-4a1f-97d1-c5da1eb9f617 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.140726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0bee6-08dd-4a1f-97d1-c5da1eb9f617 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.141214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0bee6-08dd-4a1f-97d1-c5da1eb9f617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.141322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 883/1779] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:52 2022] PUT /identity/v3/projects/8fcc970c5fe342ff9e9abd75df5be88d/users/c7ed83b38be247899991e25cb302524f/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:53.151621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.164771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.182261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:53.188766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d5a0bee6-08dd-4a1f-97d1-c5da1eb9f617 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:53.206294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5a0bee6-08dd-4a1f-97d1-c5da1eb9f617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.350571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35207ec0-9fea-4edf-ace2-95ecb1efdc7f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:53.369992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:53.372413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.374066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.374570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.378939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.404909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:53.441816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 897/1780] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:53.441816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17d93e-b7f1-4aed-a8ec-34a4e86a274e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:53.442406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17d93e-b7f1-4aed-a8ec-34a4e86a274e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:53.443375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17d93e-b7f1-4aed-a8ec-34a4e86a274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:53.582242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c17d93e-b7f1-4aed-a8ec-34a4e86a274e None None] MFA Rules not processed for user `c7ed83b38be247899991e25cb302524f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:53.583543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c17d93e-b7f1-4aed-a8ec-34a4e86a274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.666174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:53.667683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:53.695887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 884/1781] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:53.696863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:53.705584 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:53.981845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 898/1782] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:53.993342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17d93e-b7f1-4aed-a8ec-34a4e86a274e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.071013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17d93e-b7f1-4aed-a8ec-34a4e86a274e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.102661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.103557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.103700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.104051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.104843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:54.105194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.117234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.191558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3ace6-c4a9-4e9b-8d1c-c1e4575594bd service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.194203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 899/1783] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:54.195236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.216158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.217253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.217536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.217931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.231496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe52a58-a42d-4605-861e-9208b02a571e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.247376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.248076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.248369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.248651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.249355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.250700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.252260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.254544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.290809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 885/1784] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:54.291677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.335610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13c564-a946-4211-84f5-de0a8be6f942 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.359807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.360505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.360947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.361473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.362971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.365664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.411962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 900/1785] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:54.413551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.421325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:54.456283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 886/1786] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:54.466807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.647346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872feb50-2dd8-4784-a54c-5e422abcda51 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.664517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:54.730236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce25fc4-5e2d-4708-82f0-4a9b7b956438 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.750875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.752367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.753120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.753870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.762692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.771510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.772325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:54.795202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.796520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.797096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.797809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] PATH_INFO: `/v3/projects/290e7c15bbd84633872b63e39ab192f0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.809754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=290e7c15bbd84633872b63e39ab192f0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.819102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.846134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 887/1787] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/users => generated 298 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:54.853424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.883730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a5ff5-8d6a-4361-8273-e2ae5f822a02 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:54.892495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:54.893205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:54.893494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:54.893893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:54.895244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:54.897740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:54.954267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 888/1788] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:54 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:54.955752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:54.987729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0c4f5-0881-497a-bbcd-70bcedb1470b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.010920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '290e7c15bbd84633872b63e39ab192f0', 'request_id': 'req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:55.038197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '290e7c15bbd84633872b63e39ab192f0', 'request_id': 'req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:55.058748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '290e7c15bbd84633872b63e39ab192f0', 'request_id': 'req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:55.109149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.109766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.110104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.110495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.111431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.113674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.117778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] The token cache is being invalidate because project 290e7c15bbd84633872b63e39ab192f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:12:55.118737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:55.121068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:55.128353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 889/1789] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:55.132311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 901/1790] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:12:54 2022] DELETE /identity/v3/projects/290e7c15bbd84633872b63e39ab192f0 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:55.138285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.141231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:55.158069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.161857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:55.188384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64fd6f-3af7-4d9f-97ca-63042e7b1002 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.188384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279792a9-62af-42fe-b4b3-772d60f719f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.188384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279792a9-62af-42fe-b4b3-772d60f719f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.188384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279792a9-62af-42fe-b4b3-772d60f719f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279792a9-62af-42fe-b4b3-772d60f719f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.189020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279792a9-62af-42fe-b4b3-772d60f719f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.189878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279792a9-62af-42fe-b4b3-772d60f719f6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.205779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97906c-0917-4300-b6e8-45a9a4a4cb5c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.208009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 890/1791] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:55.209560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81de3-598b-4028-b455-f354f1e3cc14 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.210692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81de3-598b-4028-b455-f354f1e3cc14 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.210692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81de3-598b-4028-b455-f354f1e3cc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.212977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.215024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.215380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.215788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] PATH_INFO: `/v3/projects/d3e3f05becb5469f9e1751744023e605/users/f85f6cb5b56f46da806126a59526a10e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.238495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e3f05becb5469f9e1751744023e605, user_id=f85f6cb5b56f46da806126a59526a10e, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.241862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.302992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 902/1792] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:55 2022] PUT /identity/v3/projects/d3e3f05becb5469f9e1751744023e605/users/f85f6cb5b56f46da806126a59526a10e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:55.304895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.310733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0fc81de3-598b-4028-b455-f354f1e3cc14 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:55.337630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0fc81de3-598b-4028-b455-f354f1e3cc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:55.526678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c820e52-af16-4baa-bbe5-59bd27465e65 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.552373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.553625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.553625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.559578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.561054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.570605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.625130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 903/1793] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:55.627535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.667171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bab9c1-029c-4943-9d37-9dbb366d8d0d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.685333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff080fd-ca48-47f1-a4ae-66254e1660f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.688184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff080fd-ca48-47f1-a4ae-66254e1660f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.690022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff080fd-ca48-47f1-a4ae-66254e1660f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.693580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff080fd-ca48-47f1-a4ae-66254e1660f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.694723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff080fd-ca48-47f1-a4ae-66254e1660f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.700220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff080fd-ca48-47f1-a4ae-66254e1660f4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.726830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 891/1794] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:55.726830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc81de3-598b-4028-b455-f354f1e3cc14 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.778128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 904/1795] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:55.788632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdee8ac-4656-4624-b8d4-ea3b99d158f9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.790868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc81de3-598b-4028-b455-f354f1e3cc14 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:55.791076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdee8ac-4656-4624-b8d4-ea3b99d158f9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.791932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdee8ac-4656-4624-b8d4-ea3b99d158f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.820687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:55.822861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:55.823573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:55.825024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] PATH_INFO: `/v3/projects/094f46964f6045328c2fc15310e21f92/users/129f2490fd68404ea031eb9919e31e6f/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:55.905191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9bdee8ac-4656-4624-b8d4-ea3b99d158f9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:55.917047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9bdee8ac-4656-4624-b8d4-ea3b99d158f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:55.923234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094f46964f6045328c2fc15310e21f92, user_id=129f2490fd68404ea031eb9919e31e6f, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:55.944023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:55.990709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 905/1796] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:55.994947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdee8ac-4656-4624-b8d4-ea3b99d158f9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:55.999477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 892/1797] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:55 2022] PUT /identity/v3/projects/094f46964f6045328c2fc15310e21f92/users/129f2490fd68404ea031eb9919e31e6f/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:56.016232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:56.038731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdee8ac-4656-4624-b8d4-ea3b99d158f9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:56.110285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:56.162344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:56.204080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a903305-d882-48bd-90ef-5e25be4b8dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:56.205026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a903305-d882-48bd-90ef-5e25be4b8dff admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:56.221087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a903305-d882-48bd-90ef-5e25be4b8dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:56.221483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a903305-d882-48bd-90ef-5e25be4b8dff admin admin] PATH_INFO: `/v3/projects/d3e3f05becb5469f9e1751744023e605/users/f85f6cb5b56f46da806126a59526a10e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:56.231185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a903305-d882-48bd-90ef-5e25be4b8dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e3f05becb5469f9e1751744023e605, user_id=f85f6cb5b56f46da806126a59526a10e, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:56.235312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a903305-d882-48bd-90ef-5e25be4b8dff admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:56.273950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec31818-1dbc-4896-b583-61b99ee70639 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:56.303237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 906/1798] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:55 2022] PUT /identity/v3/projects/d3e3f05becb5469f9e1751744023e605/users/f85f6cb5b56f46da806126a59526a10e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:56.317153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161f482-f99a-4908-b044-90f3c5c0634f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:56.317560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161f482-f99a-4908-b044-90f3c5c0634f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:56.317776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161f482-f99a-4908-b044-90f3c5c0634f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:56.446619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3161f482-f99a-4908-b044-90f3c5c0634f None None] MFA Rules not processed for user `129f2490fd68404ea031eb9919e31e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:56.446619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3161f482-f99a-4908-b044-90f3c5c0634f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:56.762305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 907/1799] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:56.764084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c70ee-3c0d-4d52-9250-d03d58c20258 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:56.767212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c70ee-3c0d-4d52-9250-d03d58c20258 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:56.767698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c70ee-3c0d-4d52-9250-d03d58c20258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:56.803012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:56.804174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:56.808442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:56.810135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:56.813573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:12:56.814818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:56.826343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:56.848670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e62c70ee-3c0d-4d52-9250-d03d58c20258 None None] MFA Rules not processed for user `f85f6cb5b56f46da806126a59526a10e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:56.858361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e62c70ee-3c0d-4d52-9250-d03d58c20258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:56.864524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 893/1800] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 864 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:12:56.866158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:56.903984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5027d-2693-4d7f-9fff-7203c7e88c36 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:56.918065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:56.920287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:56.920679 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:56.921047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:56.922899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:56.930090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.017136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 894/1801] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:56 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:57.020548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.091344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 908/1802] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:57.092920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c70ee-3c0d-4d52-9250-d03d58c20258 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.177664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c70ee-3c0d-4d52-9250-d03d58c20258 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.248215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbcd8f-ea68-475f-af2c-81a23bf43873 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.260582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.261311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.262250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.262454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.263157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.265792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.338751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:57.369198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.370122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.370416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.370961 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.371576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 895/1803] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:57 2022] POST /identity/v3/projects => generated 367 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:57.372053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.376335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.376509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.379456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:57.431388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 909/1804] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:57 2022] POST /identity/v3/users => generated 289 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:57.436185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.460858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43f967-0ba7-4029-8447-b80e2414ea6e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.489918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:57.663701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdb940-32ea-4f2a-955d-2fb09cc226c0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.684667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.685696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.686405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.686910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.758241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.758960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.759332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.759515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.762200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.764305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.855553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 910/1805] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:57 2022] POST /identity/v3/users => generated 284 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:57.859822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:57.886090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127f03d-94a8-4424-abb8-b9d0664d5b02 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:57.912878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:57.919789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:57.922977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:57.923533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:57.924445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.926594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.954763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:57.959776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:57.971368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 911/1806] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:57.974084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.014152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 896/1807] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:58.015018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.047784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf1fd8-5959-4314-b67f-b71cd03f93f5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.054785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a7181-f992-4f3f-b22b-f4c0e7e607dc service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.065646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.066452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.067819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.068118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.070191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.075521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.077166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.077843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.079086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.079086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.081415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.088861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.127261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 912/1808] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:58.128546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.143033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 897/1809] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:58.144203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.207877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3e248-2064-473c-8fc0-f9df70bc0f57 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.218358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd532cae-b6d4-4db4-9c6a-c626dbb3dbed admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.229889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.231178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.231564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.231810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.232551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.235416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.236454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.239956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.240241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.240517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] PATH_INFO: `/v3/projects/8c4cc7988b994250a6875f037f32c897/users/4def03c2cc9d4c4184859d3dc2072ca7/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.283283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 913/1810] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:12:58.284978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.297707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4cc7988b994250a6875f037f32c897, user_id=4def03c2cc9d4c4184859d3dc2072ca7, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.306413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.318199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4159e-37d3-4ae0-96c0-271de4cd7f43 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.329428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac376be-5347-4464-b2da-d0d411fa36c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.330147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac376be-5347-4464-b2da-d0d411fa36c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.332655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac376be-5347-4464-b2da-d0d411fa36c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.332655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac376be-5347-4464-b2da-d0d411fa36c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.345113 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cac376be-5347-4464-b2da-d0d411fa36c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:58.361493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 898/1811] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:58 2022] PUT /identity/v3/projects/8c4cc7988b994250a6875f037f32c897/users/4def03c2cc9d4c4184859d3dc2072ca7/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:58.374274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:58.544681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b8ee9-411e-454c-a2c4-3b87672db574 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:58.571255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f6890-dcb9-41ae-ad8a-e6ae26b8de53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:58.573219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f6890-dcb9-41ae-ad8a-e6ae26b8de53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.573219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f6890-dcb9-41ae-ad8a-e6ae26b8de53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.573573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f6890-dcb9-41ae-ad8a-e6ae26b8de53 admin admin] PATH_INFO: `/v3/projects/e8d6ec4a9e654fdc89a0cc843f2993d5/users/565000189d9c46679397f6eaf5795495/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.605368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f6890-dcb9-41ae-ad8a-e6ae26b8de53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d6ec4a9e654fdc89a0cc843f2993d5, user_id=565000189d9c46679397f6eaf5795495, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.617978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f6890-dcb9-41ae-ad8a-e6ae26b8de53 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.618176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac376be-5347-4464-b2da-d0d411fa36c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:58.619471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac376be-5347-4464-b2da-d0d411fa36c8 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:58.635679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 914/1812] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:58.638102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd0e0-947a-4991-a573-f8e6c94dbf92 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.639166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd0e0-947a-4991-a573-f8e6c94dbf92 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.639719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bd0e0-947a-4991-a573-f8e6c94dbf92 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.641642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 915/1813] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:12:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:12:58.643100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.643473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.643917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.710957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 899/1814] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:12:58 2022] PUT /identity/v3/projects/e8d6ec4a9e654fdc89a0cc843f2993d5/users/565000189d9c46679397f6eaf5795495/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:58.716749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdd97-fdb2-4bb2-9b94-1bfb7c03d414 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.717757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdd97-fdb2-4bb2-9b94-1bfb7c03d414 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.718070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1bdd97-fdb2-4bb2-9b94-1bfb7c03d414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.735130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] MFA Rules not processed for user `4def03c2cc9d4c4184859d3dc2072ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:58.738647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:58.770292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3c1bdd97-fdb2-4bb2-9b94-1bfb7c03d414 None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:58.771728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3c1bdd97-fdb2-4bb2-9b94-1bfb7c03d414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:58.815847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:58.849953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:58.951818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 900/1815] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:12:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:58.977199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbab78-cc84-4489-91bb-f6bb5f372639 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:58.977544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbab78-cc84-4489-91bb-f6bb5f372639 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:58.978406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbab78-cc84-4489-91bb-f6bb5f372639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:58.992122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 916/1816] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:58.997939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.004634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.066731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f7632-6374-4bb8-b759-5b3327e1ebed None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.084406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.085058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.085471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.085990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.125654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-afdbab78-cc84-4489-91bb-f6bb5f372639 None None] MFA Rules not processed for user `565000189d9c46679397f6eaf5795495`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:59.126362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.131776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.146544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-afdbab78-cc84-4489-91bb-f6bb5f372639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.170534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 917/1817] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:12:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:12:59.174372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.452985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2d0bd-c219-42e7-84e4-216c8917a490 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.460087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 901/1818] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:12:59.462609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbab78-cc84-4489-91bb-f6bb5f372639 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.469421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.471498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.471819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.472064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.472779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.479293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.502073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbab78-cc84-4489-91bb-f6bb5f372639 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.515708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.517770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.519277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.519957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] PATH_INFO: `/v3/users/c7ed83b38be247899991e25cb302524f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.524302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ed83b38be247899991e25cb302524f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.539219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.572182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 918/1819] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/projects => generated 377 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:59.573364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.711194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ed83b38be247899991e25cb302524f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.712181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ed83b38be247899991e25cb302524f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.724003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ed83b38be247899991e25cb302524f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.724266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d791fa-88cf-44e2-9d15-b8d5899f07c6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.730791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ed83b38be247899991e25cb302524f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.736701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c78a319-8050-46c6-b002-f22c763273a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:12:59.739971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a319-8050-46c6-b002-f22c763273a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.740262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a319-8050-46c6-b002-f22c763273a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.740538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78a319-8050-46c6-b002-f22c763273a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.741407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78a319-8050-46c6-b002-f22c763273a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:12:59.747041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c78a319-8050-46c6-b002-f22c763273a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:12:59.749032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ed83b38be247899991e25cb302524f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:12:59.807486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 902/1820] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:12:59 2022] DELETE /identity/v3/users/c7ed83b38be247899991e25cb302524f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:12:59.809300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:12:59.811563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.826732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 919/1821] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/projects => generated 367 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:12:59.827947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5fc7-6129-455b-96c6-3117abcac6db None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:12:59.828245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5fc7-6129-455b-96c6-3117abcac6db None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:12:59.828540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c5fc7-6129-455b-96c6-3117abcac6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:12:59.831565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:12:59.869782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1c8c5fc7-6129-455b-96c6-3117abcac6db None None] MFA Rules not processed for user `bfbcc9b45bc341228bcc0f2192040224`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:12:59.871007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c8c5fc7-6129-455b-96c6-3117abcac6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:12:59.922392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf725e5-45b5-4212-89d8-30fec8fd120e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:12:59.924667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:00.027824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 920/1822] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:00.035749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.036047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.036335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.051079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.052616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.052912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.053386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.054087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.056091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.077264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:00.080140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.094476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 921/1823] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:00.095325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.107004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.123586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e8d7d-0629-4ef4-b4fe-e23503e0f54e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.125313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 903/1824] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:12:59 2022] POST /identity/v3/users => generated 295 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:00.126314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.129949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.134056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.144065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.145606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.146373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.147275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.149828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.177423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 922/1825] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/users => generated 290 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:00.178592 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.193886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c7afb-9088-4002-99d6-e6b502d6bedf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.201293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.203087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.203582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.203984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.204859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.206892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.222151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 923/1826] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:00.223749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.245405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c6f03-9811-4548-9497-568e946de1c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.252316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.252548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.252901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.253879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.254875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.257420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.274476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 924/1827] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:00.275719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.290611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d094d77-f5bb-40f0-bc14-22cef9ae4727 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.301864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.303082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.303269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.303552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] PATH_INFO: `/v3/projects/d1f8ef3d1c144a6688c838c16e4a4892/users/ca926268f2484010b1426f75bec6b552/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.312272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181e2de-d8cd-478a-bad3-bcf7e66f5d63 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.318119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.322777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.323224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.323712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.331928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f8ef3d1c144a6688c838c16e4a4892, user_id=ca926268f2484010b1426f75bec6b552, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.336020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.346373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.348644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.370368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 925/1828] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:00 2022] PUT /identity/v3/projects/d1f8ef3d1c144a6688c838c16e4a4892/users/ca926268f2484010b1426f75bec6b552/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:00.371201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.411642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 904/1829] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:00.413020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.437572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4883a67-581b-461e-8ca2-6017f19987ae service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.483409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83319a-ece1-4623-b98d-a3f351e262db admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.491939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.494590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.494981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.496501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] PATH_INFO: `/v3/projects/460ecb0e1c054c429067456fe3d4ea0c/users/2c136277f4f74bfda5766bd29fb0ee1d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.558320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460ecb0e1c054c429067456fe3d4ea0c, user_id=2c136277f4f74bfda5766bd29fb0ee1d, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.561646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.562848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:00.589185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 926/1830] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:00 2022] PUT /identity/v3/projects/460ecb0e1c054c429067456fe3d4ea0c/users/2c136277f4f74bfda5766bd29fb0ee1d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:00.590523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.591890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.592042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.592730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.593072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.597037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.599196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.615829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 905/1831] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:00.625427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.670607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062a674-3e14-4476-81e5-9f7bc1afce37 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.688246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:00.716897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ea8e4-ee12-4793-8796-56f8c27f7558 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.723068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.724163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.724602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.724921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.725814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.728216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.763692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 927/1832] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:00.764817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.785083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf890ad-843c-4917-80fc-e33493f34da6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:00.799384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6840721-fc8e-4b9f-b490-19bb0f7f1207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.800519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6840721-fc8e-4b9f-b490-19bb0f7f1207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.800913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6840721-fc8e-4b9f-b490-19bb0f7f1207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.801471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6840721-fc8e-4b9f-b490-19bb0f7f1207 admin admin] PATH_INFO: `/v3/projects/460ecb0e1c054c429067456fe3d4ea0c/users/2c136277f4f74bfda5766bd29fb0ee1d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.803798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6840721-fc8e-4b9f-b490-19bb0f7f1207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460ecb0e1c054c429067456fe3d4ea0c, user_id=2c136277f4f74bfda5766bd29fb0ee1d, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.809029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6840721-fc8e-4b9f-b490-19bb0f7f1207 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.837200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 928/1833] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:00 2022] PUT /identity/v3/projects/460ecb0e1c054c429067456fe3d4ea0c/users/2c136277f4f74bfda5766bd29fb0ee1d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:00.838716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.839126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.839564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.871806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1cc3c-b5ac-4527-bc2f-de5c3e0bad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:00.872852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1cc3c-b5ac-4527-bc2f-de5c3e0bad36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.873057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] MFA Rules not processed for user `2c136277f4f74bfda5766bd29fb0ee1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:00.873171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1cc3c-b5ac-4527-bc2f-de5c3e0bad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.873422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1cc3c-b5ac-4527-bc2f-de5c3e0bad36 admin admin] PATH_INFO: `/v3/projects/d1f8ef3d1c144a6688c838c16e4a4892/users/ca926268f2484010b1426f75bec6b552/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.874790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.875517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1cc3c-b5ac-4527-bc2f-de5c3e0bad36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f8ef3d1c144a6688c838c16e4a4892, user_id=ca926268f2484010b1426f75bec6b552, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:00.878478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1cc3c-b5ac-4527-bc2f-de5c3e0bad36 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:00.904771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 906/1834] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:00 2022] PUT /identity/v3/projects/d1f8ef3d1c144a6688c838c16e4a4892/users/ca926268f2484010b1426f75bec6b552/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:00.908379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:00.908670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:00.908952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:00.962670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] MFA Rules not processed for user `ca926268f2484010b1426f75bec6b552`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:00.964736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:00.972063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 929/1835] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:00.987960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:00.993803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.102232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 907/1836] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:01.103918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.109328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8bd0f-e6db-4f07-98e0-dd26af7ee7fa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.109790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.213058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0310079-959b-4a46-8249-a9c0dee23250 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.218266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.219057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.219412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.219765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.228618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:01.250217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:01.254214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.255224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.255704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.256454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] PATH_INFO: `/v3/projects/8fcc970c5fe342ff9e9abd75df5be88d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.259436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fcc970c5fe342ff9e9abd75df5be88d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.262802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.317702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.319549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.337288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 908/1837] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:01.338835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.341113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.343886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fcc970c5fe342ff9e9abd75df5be88d', 'request_id': 'req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.356767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89e4d3-91df-4146-9355-5aee830aef4d service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.360516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fcc970c5fe342ff9e9abd75df5be88d', 'request_id': 'req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.374807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fcc970c5fe342ff9e9abd75df5be88d', 'request_id': 'req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.406589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] The token cache is being invalidate because project 8fcc970c5fe342ff9e9abd75df5be88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:01.406964 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.407909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0cc07ca-829e-4c0c-87c8-69af80a6cb88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.410049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 930/1838] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:00 2022] DELETE /identity/v3/projects/8fcc970c5fe342ff9e9abd75df5be88d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:01.412068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b3133-097c-4796-bbe1-b932604400f6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.412068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b3133-097c-4796-bbe1-b932604400f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.412372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b3133-097c-4796-bbe1-b932604400f6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.413974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 931/1839] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:13:01.414821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b3133-097c-4796-bbe1-b932604400f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.416859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b1b3133-097c-4796-bbe1-b932604400f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.477286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4396cf1-6579-4174-8562-13ca3b4e4348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.478170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396cf1-6579-4174-8562-13ca3b4e4348 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.478532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396cf1-6579-4174-8562-13ca3b4e4348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.479579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396cf1-6579-4174-8562-13ca3b4e4348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.484560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4396cf1-6579-4174-8562-13ca3b4e4348 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.577843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b3133-097c-4796-bbe1-b932604400f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.588931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.595260 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.596169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.598157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] PATH_INFO: `/v3/users/f85f6cb5b56f46da806126a59526a10e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.599719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f85f6cb5b56f46da806126a59526a10e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.601910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.657337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4396cf1-6579-4174-8562-13ca3b4e4348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.658866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4396cf1-6579-4174-8562-13ca3b4e4348 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:01.677710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f85f6cb5b56f46da806126a59526a10e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.678917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f85f6cb5b56f46da806126a59526a10e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.679777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 909/1840] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:01.681729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.682340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.682728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.689342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f85f6cb5b56f46da806126a59526a10e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.704637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f85f6cb5b56f46da806126a59526a10e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.719723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:01.721761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.729081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f85f6cb5b56f46da806126a59526a10e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:01.761931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 932/1841] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:01 2022] DELETE /identity/v3/users/f85f6cb5b56f46da806126a59526a10e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:01.762955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:01.768221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.785493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:01.818191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:01.839343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 910/1842] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:13:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:01.865853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31c206-f876-4830-8186-2dc22b38c5c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:01.871116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:01.871955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:01.872302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:01.872613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:01.875220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:01.973226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:01.976361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:02.026671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 933/1843] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:02.189254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:02.190886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:02.231676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13d58f-f947-4f8e-925a-f301fcddb0de None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:02.289908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:02.292566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:02.346230 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:02.347293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:02.347764 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:02.348344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] PATH_INFO: `/v3/projects/d3e3f05becb5469f9e1751744023e605` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:02.350994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e3f05becb5469f9e1751744023e605)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:02.353556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:02.417363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e3b3e-e0d5-4bdc-9652-014d1ab8ec70 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:02.425620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56504b5-3eca-44f0-836a-d40f34fdd0e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:02.426659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56504b5-3eca-44f0-836a-d40f34fdd0e6 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:02.427003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56504b5-3eca-44f0-836a-d40f34fdd0e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:02.427294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56504b5-3eca-44f0-836a-d40f34fdd0e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:02.441040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e3f05becb5469f9e1751744023e605', 'request_id': 'req-f262c493-bab4-4b64-973e-2cd56c640326'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:02.443641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56504b5-3eca-44f0-836a-d40f34fdd0e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:02.447583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56504b5-3eca-44f0-836a-d40f34fdd0e6 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:02.454720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e3f05becb5469f9e1751744023e605', 'request_id': 'req-f262c493-bab4-4b64-973e-2cd56c640326'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:02.465530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e3f05becb5469f9e1751744023e605', 'request_id': 'req-f262c493-bab4-4b64-973e-2cd56c640326'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:02.466119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 934/1844] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:13:02.494952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] The token cache is being invalidate because project d3e3f05becb5469f9e1751744023e605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:02.494952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:02.494952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f262c493-bab4-4b64-973e-2cd56c640326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:02.499165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 911/1845] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:02 2022] DELETE /identity/v3/projects/d3e3f05becb5469f9e1751744023e605 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:02.504025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38396f-8341-4674-8d82-88606b64f656 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:02.504517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38396f-8341-4674-8d82-88606b64f656 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:02.505003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38396f-8341-4674-8d82-88606b64f656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:02.532531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e38396f-8341-4674-8d82-88606b64f656 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:02.534001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e38396f-8341-4674-8d82-88606b64f656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:02.639486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:02.707500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 935/1846] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:02.709588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2db73f-87e1-4988-a716-68edcd28cb5f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:02.710158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2db73f-87e1-4988-a716-68edcd28cb5f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:02.710310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2db73f-87e1-4988-a716-68edcd28cb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:02.742869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2b2db73f-87e1-4988-a716-68edcd28cb5f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:02.744804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2b2db73f-87e1-4988-a716-68edcd28cb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:02.758785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 912/1847] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:02.764727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38396f-8341-4674-8d82-88606b64f656 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:02.782872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38396f-8341-4674-8d82-88606b64f656 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:02.800030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:02.802857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:02.812742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:02.812742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:02.812742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:02.812742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:02.812742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:02.844610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 936/1848] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:02.853661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2db73f-87e1-4988-a716-68edcd28cb5f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:02.864902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2db73f-87e1-4988-a716-68edcd28cb5f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:02.875752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:02.876442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:02.876729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:02.877431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:02.878179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:02.881061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:02.921656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 913/1849] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:02 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:02.927744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:02.941354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46b2fe-0bab-4a2f-899c-d731c6b42029 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.037679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.038818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.039119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.039397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.040289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.042377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.081935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 937/1850] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:02 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:03.086539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.106748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc66544-6de4-47f1-bef6-2f5df4ba54b2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.115076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.115777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.116184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.116570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.117443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.118988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.131932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 914/1851] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:03.135766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.147746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05021ea4-ded2-414a-af09-f490384382d5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.154324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b0f86-c783-42f4-ad6d-dbabea170312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.155162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b0f86-c783-42f4-ad6d-dbabea170312 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.156246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b0f86-c783-42f4-ad6d-dbabea170312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.156246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b0f86-c783-42f4-ad6d-dbabea170312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.156740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b0f86-c783-42f4-ad6d-dbabea170312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.158580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b0f86-c783-42f4-ad6d-dbabea170312 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.174182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 938/1852] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:03.175853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.187816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4504c-db3f-4bed-b660-4b89be052c90 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.194668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.196378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.196863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.197314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] PATH_INFO: `/v3/projects/dac369a021ea484fb12cc8e8f586ab54/users/bae9ecea988c4a13ad38fc3312eedbc9/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.224086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac369a021ea484fb12cc8e8f586ab54, user_id=bae9ecea988c4a13ad38fc3312eedbc9, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.227032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.241884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 915/1853] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:03 2022] PUT /identity/v3/projects/dac369a021ea484fb12cc8e8f586ab54/users/bae9ecea988c4a13ad38fc3312eedbc9/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:03.256132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.256771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.257092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.279665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:03.280959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:03.504564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 939/1854] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:03.515175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.517359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:03.552244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:03.595059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad2ebc-b373-4414-a081-5c09d5df877c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.599608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d858e8-934f-4224-9f92-bb186af38f70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.600281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d858e8-934f-4224-9f92-bb186af38f70 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.600590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d858e8-934f-4224-9f92-bb186af38f70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.600854 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d858e8-934f-4224-9f92-bb186af38f70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.614816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d858e8-934f-4224-9f92-bb186af38f70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.616458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d858e8-934f-4224-9f92-bb186af38f70 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.630615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 916/1855] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:03.640628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:03.642153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:03.660455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc692a6-5124-4f72-91ae-2cace5f1de27 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:03.668332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:03.670415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.670983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.671522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] PATH_INFO: `/v3/users/565000189d9c46679397f6eaf5795495` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.675691 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=565000189d9c46679397f6eaf5795495)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:03.678059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:03.754814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:03.761646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '565000189d9c46679397f6eaf5795495'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.763566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '565000189d9c46679397f6eaf5795495'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.766275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '565000189d9c46679397f6eaf5795495'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.780654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '565000189d9c46679397f6eaf5795495'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.807111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '565000189d9c46679397f6eaf5795495'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:03.850406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 940/1856] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:03 2022] DELETE /identity/v3/users/565000189d9c46679397f6eaf5795495 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:03.855605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8132665-f3de-4b60-8c00-d1b90f74adcb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:03.856702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8132665-f3de-4b60-8c00-d1b90f74adcb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:03.857072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8132665-f3de-4b60-8c00-d1b90f74adcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:03.891187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e8132665-f3de-4b60-8c00-d1b90f74adcb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:03.893400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e8132665-f3de-4b60-8c00-d1b90f74adcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:03.994040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 917/1857] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:04.037795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:04.039299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:04.181062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c0d0e-de71-4240-9cfd-555f63c252ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:04.186159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:04.187392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:04.187918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:04.188488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:04.203277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:04.203277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:04.217001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 941/1858] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:04.835767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:04.836144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:04.836434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:04.861962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:04.864071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:04.886265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78709cc6-6aa2-4200-92c5-9378605da3f7 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:04.916211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:04.917413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:04.918595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:04.919430 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] PATH_INFO: `/v3/projects/e8d6ec4a9e654fdc89a0cc843f2993d5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:04.923101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d6ec4a9e654fdc89a0cc843f2993d5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:04.923785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] MFA Rules not processed for user `bae9ecea988c4a13ad38fc3312eedbc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:04.926526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:04.927583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.042498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d6ec4a9e654fdc89a0cc843f2993d5', 'request_id': 'req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.076723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d6ec4a9e654fdc89a0cc843f2993d5', 'request_id': 'req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.078439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 918/1859] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:05.096400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d6ec4a9e654fdc89a0cc843f2993d5', 'request_id': 'req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.098276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.103998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.125524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] The token cache is being invalidate because project e8d6ec4a9e654fdc89a0cc843f2993d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:05.125959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.127280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.130696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 942/1860] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:04 2022] DELETE /identity/v3/projects/e8d6ec4a9e654fdc89a0cc843f2993d5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:05.134698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.137535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.157188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74617cba-5a72-4a9f-9d8b-9b7589419be2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.245223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:05.255004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:05.255689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.255970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.256314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.256946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:05.259361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:05.289383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:05.311882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 919/1861] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:05 2022] POST /identity/v3/projects => generated 373 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:05.316841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae58cfb-da1a-4dfc-81e9-63fe09d6e507 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.322897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.327105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.486335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:05.487384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.487672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.488081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] PATH_INFO: `/v3/users/2c136277f4f74bfda5766bd29fb0ee1d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.490473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c136277f4f74bfda5766bd29fb0ee1d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:05.492648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:05.510399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7917-92a3-4043-937f-42c66809a514 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.525075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:05.525787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.526079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.526360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.527425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:05.530056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:05.582317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 920/1862] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:05 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:05.590200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.592159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c136277f4f74bfda5766bd29fb0ee1d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.592865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c136277f4f74bfda5766bd29fb0ee1d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.593509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c136277f4f74bfda5766bd29fb0ee1d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.603975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee292a3-b590-4290-bfc4-78a4114f3109 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.612869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:05.613573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.613958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.614232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.615152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c136277f4f74bfda5766bd29fb0ee1d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.615319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:05.617232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:05.629898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 921/1863] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:05.630216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c136277f4f74bfda5766bd29fb0ee1d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:05.633818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.635683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:05.651006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56cf7ab-f46d-47d1-8a71-7219a2182366 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.656339 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:05.657339 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:05.657691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:05.658072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] PATH_INFO: `/v3/projects/4ac04c61743b408ba01b92a225fdf1c8/users/1c25775db8e44a2ba8523597461106ff/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:05.664774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 943/1864] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:05 2022] DELETE /identity/v3/users/2c136277f4f74bfda5766bd29fb0ee1d => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:05.684089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac04c61743b408ba01b92a225fdf1c8, user_id=1c25775db8e44a2ba8523597461106ff, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:05.689740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:05.717765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 922/1865] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:05 2022] PUT /identity/v3/projects/4ac04c61743b408ba01b92a225fdf1c8/users/1c25775db8e44a2ba8523597461106ff/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:05.720552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:05.804764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b7a8-67dd-4062-af39-c8b2f045e438 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:05.822355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:06.033358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.034576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.034576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.034576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.034984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.036699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.062060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 944/1866] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:06.063722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.085323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8a2c-b38e-4acf-9284-87333ec7f10c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.104853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d8c13-b648-43c8-b383-66194bcb0c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.106443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d8c13-b648-43c8-b383-66194bcb0c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.106891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d8c13-b648-43c8-b383-66194bcb0c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.107410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d8c13-b648-43c8-b383-66194bcb0c43 admin admin] PATH_INFO: `/v3/projects/4ac04c61743b408ba01b92a225fdf1c8/users/1c25775db8e44a2ba8523597461106ff/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.116560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d8c13-b648-43c8-b383-66194bcb0c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac04c61743b408ba01b92a225fdf1c8, user_id=1c25775db8e44a2ba8523597461106ff, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.120087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d8c13-b648-43c8-b383-66194bcb0c43 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.145437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 923/1867] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:06 2022] PUT /identity/v3/projects/4ac04c61743b408ba01b92a225fdf1c8/users/1c25775db8e44a2ba8523597461106ff/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:06.211220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.213702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.235220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32072a0-845b-44ff-97a4-bfadb9254c3b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.330833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.331827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.332189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.332541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] PATH_INFO: `/v3/projects/460ecb0e1c054c429067456fe3d4ea0c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.333936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=460ecb0e1c054c429067456fe3d4ea0c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.336284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.406811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460ecb0e1c054c429067456fe3d4ea0c', 'request_id': 'req-4655ad62-7a20-4d60-88b8-9c2f54da3f48'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.415872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460ecb0e1c054c429067456fe3d4ea0c', 'request_id': 'req-4655ad62-7a20-4d60-88b8-9c2f54da3f48'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.426719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460ecb0e1c054c429067456fe3d4ea0c', 'request_id': 'req-4655ad62-7a20-4d60-88b8-9c2f54da3f48'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.438559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] The token cache is being invalidate because project 460ecb0e1c054c429067456fe3d4ea0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:06.438997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.439899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4655ad62-7a20-4d60-88b8-9c2f54da3f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:06.441687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 945/1868] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:06 2022] DELETE /identity/v3/projects/460ecb0e1c054c429067456fe3d4ea0c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:06.446560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917d08d-2b91-4795-931c-3cec51881352 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.446856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917d08d-2b91-4795-931c-3cec51881352 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.447144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917d08d-2b91-4795-931c-3cec51881352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.467105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4917d08d-2b91-4795-931c-3cec51881352 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:06.468330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4917d08d-2b91-4795-931c-3cec51881352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.675444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 924/1869] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:06.684011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14097888-21f1-4cf3-b14f-963885e63033 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.684396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14097888-21f1-4cf3-b14f-963885e63033 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.684752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14097888-21f1-4cf3-b14f-963885e63033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.704403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-14097888-21f1-4cf3-b14f-963885e63033 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:06.705916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-14097888-21f1-4cf3-b14f-963885e63033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:06.715469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 946/1870] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:06.720258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917d08d-2b91-4795-931c-3cec51881352 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.731747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917d08d-2b91-4795-931c-3cec51881352 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.736942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.737707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.738020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.738296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.739088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:06.739331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.741239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.764412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 925/1871] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:13:06.768773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14097888-21f1-4cf3-b14f-963885e63033 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.779661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14097888-21f1-4cf3-b14f-963885e63033 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.784596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.785952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.786223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.786496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.787198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.789121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:06.845327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 947/1872] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:06 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:06.849795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:06.864411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148fab3-c409-4a12-a298-be3a3de98a94 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:06.881299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:06.989777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:06.990474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:06.990931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:06.991259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:06.992201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:06.994614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.021805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 926/1873] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:06 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:07.025271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.044474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca78b5f-5177-4daf-8781-65f842edb36a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.050318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02472-8d63-448e-af08-4f00387f4af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.051192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02472-8d63-448e-af08-4f00387f4af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.051479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02472-8d63-448e-af08-4f00387f4af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.051758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02472-8d63-448e-af08-4f00387f4af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.052492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff02472-8d63-448e-af08-4f00387f4af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.054352 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff02472-8d63-448e-af08-4f00387f4af5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.069354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 948/1874] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:07.074282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.087623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0ab5d-2926-4209-a60e-7ae2d4ac1a3e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.101435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.106466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.106466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.113128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.116501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.118241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.135492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 927/1875] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:07.144116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.144448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.147987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.149245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.162178 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e8323-fcd3-4bbb-9534-483832686e6e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.168629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67154e6d-7929-4fca-ae26-980b4bc0563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.169844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154e6d-7929-4fca-ae26-980b4bc0563a admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.170290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154e6d-7929-4fca-ae26-980b4bc0563a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.170745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67154e6d-7929-4fca-ae26-980b4bc0563a admin admin] PATH_INFO: `/v3/projects/e8c5ccd628a1482ea6fbb6f612cce56d/users/cd4207d1f42c46dfa9c2bb13d3808749/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.180576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:07.191766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67154e6d-7929-4fca-ae26-980b4bc0563a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c5ccd628a1482ea6fbb6f612cce56d, user_id=cd4207d1f42c46dfa9c2bb13d3808749, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.197173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67154e6d-7929-4fca-ae26-980b4bc0563a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.223282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 928/1876] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:07 2022] PUT /identity/v3/projects/e8c5ccd628a1482ea6fbb6f612cce56d/users/cd4207d1f42c46dfa9c2bb13d3808749/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:07.233503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] MFA Rules not processed for user `1c25775db8e44a2ba8523597461106ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:07.233503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.237141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2f18-96a4-40fa-bbca-53e05ceb0ccc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.237692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2f18-96a4-40fa-bbca-53e05ceb0ccc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.238155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2f18-96a4-40fa-bbca-53e05ceb0ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.284527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f50f2f18-96a4-40fa-bbca-53e05ceb0ccc None None] MFA Rules not processed for user `cd4207d1f42c46dfa9c2bb13d3808749`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:07.285778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f50f2f18-96a4-40fa-bbca-53e05ceb0ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.332163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 949/1877] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:07.345642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.348138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.396566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 929/1878] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:07.409403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f2f18-96a4-40fa-bbca-53e05ceb0ccc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.495997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df12a46-c948-4179-83ca-ecb7717d08f6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.507571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.508764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.509457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.510454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.514345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:07.517344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:07.523916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f2f18-96a4-40fa-bbca-53e05ceb0ccc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.533219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.534627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.535728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.536436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.537991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.541774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.610984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 930/1879] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:07.612271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.655177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.657085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.669152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 950/1880] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:13:07.748493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1f316-79eb-4bee-ac5a-493e8efb45c3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.761778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.763161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.763869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.766087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.767615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.773454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.819435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 931/1881] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:07 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:07.825090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.854286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e6fac-fa26-4d29-a46f-0eafe008ad38 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.862642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.863816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.864396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.864941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.866316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.869458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:07.893071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 951/1882] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:07.899689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:07.931829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0e342-4e68-4c4b-af07-2459200f337b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:07.939875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:07.941966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:07.943280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:07.944931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] PATH_INFO: `/v3/projects/116d0dc088614961bcbc9b7dde44a6c7/users/042cbe9644b341e1b460ff7e4c0cf403/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:07.976272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116d0dc088614961bcbc9b7dde44a6c7, user_id=042cbe9644b341e1b460ff7e4c0cf403, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:07.979660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.010872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 932/1883] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:07 2022] PUT /identity/v3/projects/116d0dc088614961bcbc9b7dde44a6c7/users/042cbe9644b341e1b460ff7e4c0cf403/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:08.015695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.037642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae009cd-f11e-4f9a-91ac-f9be10ac0e2c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.092791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:08.157080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295770a-5598-4fc3-9faa-fef3e2709082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.158248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295770a-5598-4fc3-9faa-fef3e2709082 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.161233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295770a-5598-4fc3-9faa-fef3e2709082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.161808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295770a-5598-4fc3-9faa-fef3e2709082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.162805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295770a-5598-4fc3-9faa-fef3e2709082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.171106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295770a-5598-4fc3-9faa-fef3e2709082 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.197429 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 952/1884] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:08.201207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.215118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a588835-478d-4aa4-a292-29477c176a14 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.234594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.235850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.236448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.238605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] PATH_INFO: `/v3/projects/116d0dc088614961bcbc9b7dde44a6c7/users/042cbe9644b341e1b460ff7e4c0cf403/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.249084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116d0dc088614961bcbc9b7dde44a6c7, user_id=042cbe9644b341e1b460ff7e4c0cf403, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.254168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.273819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 933/1885] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:08 2022] PUT /identity/v3/projects/116d0dc088614961bcbc9b7dde44a6c7/users/042cbe9644b341e1b460ff7e4c0cf403/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:08.286939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488ce4c-c2d2-45e8-b550-4709e76085f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.287753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488ce4c-c2d2-45e8-b550-4709e76085f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.288344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a488ce4c-c2d2-45e8-b550-4709e76085f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.334247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a488ce4c-c2d2-45e8-b550-4709e76085f7 None None] MFA Rules not processed for user `042cbe9644b341e1b460ff7e4c0cf403`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:08.336089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a488ce4c-c2d2-45e8-b550-4709e76085f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:08.524090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 953/1886] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:08.538712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:08.685196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26494244-0c05-4068-9f2d-d107e172ae01 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:08.690969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531bbb6-91c9-47f3-b3de-aa5233382257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:08.693029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531bbb6-91c9-47f3-b3de-aa5233382257 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:08.693323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531bbb6-91c9-47f3-b3de-aa5233382257 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:08.693883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531bbb6-91c9-47f3-b3de-aa5233382257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:08.706942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531bbb6-91c9-47f3-b3de-aa5233382257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:08.709944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531bbb6-91c9-47f3-b3de-aa5233382257 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:08.740496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 934/1887] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:09.546045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488ce4c-c2d2-45e8-b550-4709e76085f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:09.628427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a488ce4c-c2d2-45e8-b550-4709e76085f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.633275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:09.634359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.634615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.635379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] PATH_INFO: `/v3/users/cd4207d1f42c46dfa9c2bb13d3808749` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.638629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4207d1f42c46dfa9c2bb13d3808749)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:09.640676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:09.694386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4207d1f42c46dfa9c2bb13d3808749'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.695257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4207d1f42c46dfa9c2bb13d3808749'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.695902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4207d1f42c46dfa9c2bb13d3808749'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.705472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4207d1f42c46dfa9c2bb13d3808749'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.711867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:09.716282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4207d1f42c46dfa9c2bb13d3808749'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:09.741200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 954/1888] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:09 2022] DELETE /identity/v3/users/cd4207d1f42c46dfa9c2bb13d3808749 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:09.746274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.746796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.746917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.764229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:09.765562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.798720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:09.830499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 935/1889] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:09.840995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:09.842547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:09.909895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ddb71c-e0de-4cc7-8c08-8566a3bd8383 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:09.913782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:09.914451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:09.914741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:09.915022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:09.926539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:09.928014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:09.939516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 955/1890] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:10.493831 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:10.495483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:10.618004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe68cf-f8d0-40b4-ac57-99dc123b2077 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:10.623378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f8f2cf-6646-4a1f-a0da-3508820e991c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:10.624091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8f2cf-6646-4a1f-a0da-3508820e991c service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:10.624380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8f2cf-6646-4a1f-a0da-3508820e991c service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:10.624659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f8f2cf-6646-4a1f-a0da-3508820e991c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:10.626818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-63f8f2cf-6646-4a1f-a0da-3508820e991c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:10.796155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8f2cf-6646-4a1f-a0da-3508820e991c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:10.798502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f8f2cf-6646-4a1f-a0da-3508820e991c service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:10.822523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 936/1891] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:11.112383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.115290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.145593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72993e2-be54-4b54-aa7c-16ababf58e3c service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.155092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.157043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.157535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.158075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] PATH_INFO: `/v3/users/129f2490fd68404ea031eb9919e31e6f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.160626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=129f2490fd68404ea031eb9919e31e6f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.163977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.255552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '129f2490fd68404ea031eb9919e31e6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.257865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '129f2490fd68404ea031eb9919e31e6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.259521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '129f2490fd68404ea031eb9919e31e6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.278681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '129f2490fd68404ea031eb9919e31e6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.279500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '129f2490fd68404ea031eb9919e31e6f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.301216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 956/1892] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:11 2022] DELETE /identity/v3/users/129f2490fd68404ea031eb9919e31e6f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:11.306170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.306478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.306827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.326953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:11.328191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.400319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 937/1893] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:11.411709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.413192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.463283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.465042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.482273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531536c1-475c-403d-922c-27ef1f81fb3c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.491899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.493857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.494135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.494434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] PATH_INFO: `/v3/projects/e8c5ccd628a1482ea6fbb6f612cce56d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.496303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c5ccd628a1482ea6fbb6f612cce56d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.498503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.518076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535ac63-1fc4-4b01-be06-f19f05ca3bc6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.523677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.524992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.525490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.526434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.552728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:11.566595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:11.583455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.585783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.602799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c5ccd628a1482ea6fbb6f612cce56d', 'request_id': 'req-1d30a8fb-cd29-44d6-996e-2872400efb20'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.606825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 957/1894] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:11.619379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c5ccd628a1482ea6fbb6f612cce56d', 'request_id': 'req-1d30a8fb-cd29-44d6-996e-2872400efb20'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.636361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c5ccd628a1482ea6fbb6f612cce56d', 'request_id': 'req-1d30a8fb-cd29-44d6-996e-2872400efb20'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.672243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] The token cache is being invalidate because project e8c5ccd628a1482ea6fbb6f612cce56d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:11.672811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.674998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.679100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 938/1895] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:11 2022] DELETE /identity/v3/projects/e8c5ccd628a1482ea6fbb6f612cce56d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:11.685703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.704971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50745bc0-40a9-4ba2-a27c-145fcd4f4868 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:11.818998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:11.820171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:11.820457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:11.820800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] PATH_INFO: `/v3/users/042cbe9644b341e1b460ff7e4c0cf403` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:11.822409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042cbe9644b341e1b460ff7e4c0cf403)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:11.824519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:11.885321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042cbe9644b341e1b460ff7e4c0cf403'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.886152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042cbe9644b341e1b460ff7e4c0cf403'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.886771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042cbe9644b341e1b460ff7e4c0cf403'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.909579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042cbe9644b341e1b460ff7e4c0cf403'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.924670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042cbe9644b341e1b460ff7e4c0cf403'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:11.950942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 958/1896] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:11 2022] DELETE /identity/v3/users/042cbe9644b341e1b460ff7e4c0cf403 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:11.954818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:11.956490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:11.973888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30a8fb-cd29-44d6-996e-2872400efb20 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.061157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.063085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.063588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.064029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] PATH_INFO: `/v3/projects/094f46964f6045328c2fc15310e21f92` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.065828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=094f46964f6045328c2fc15310e21f92)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.069285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.190506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094f46964f6045328c2fc15310e21f92', 'request_id': 'req-7424375a-8219-46e8-a2ab-3e21332525bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.209825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094f46964f6045328c2fc15310e21f92', 'request_id': 'req-7424375a-8219-46e8-a2ab-3e21332525bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.227601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094f46964f6045328c2fc15310e21f92', 'request_id': 'req-7424375a-8219-46e8-a2ab-3e21332525bb'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.263997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] The token cache is being invalidate because project 094f46964f6045328c2fc15310e21f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:12.265146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.276635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.282681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 939/1897] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:11 2022] DELETE /identity/v3/projects/094f46964f6045328c2fc15310e21f92 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:12.287064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.289007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:12.327256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712de2b5-808d-441e-b66a-5ec8393f45f5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.420190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:12.421701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:12.507128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.508103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.508396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.508679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] PATH_INFO: `/v3/users/4def03c2cc9d4c4184859d3dc2072ca7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.513394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4def03c2cc9d4c4184859d3dc2072ca7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.518801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.523041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:12.553005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:12.604276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424375a-8219-46e8-a2ab-3e21332525bb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:12.625862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:12.627820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.628550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.629076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] PATH_INFO: `/v3/projects/116d0dc088614961bcbc9b7dde44a6c7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:12.633408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] RBAC: Authorizing `identity:delete_project(project_id=116d0dc088614961bcbc9b7dde44a6c7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:12.665129 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4def03c2cc9d4c4184859d3dc2072ca7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.665738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:12.667756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4def03c2cc9d4c4184859d3dc2072ca7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.669764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4def03c2cc9d4c4184859d3dc2072ca7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.689303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4def03c2cc9d4c4184859d3dc2072ca7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.709445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9548f008-79e2-413f-84fc-7b7f0f526f82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4def03c2cc9d4c4184859d3dc2072ca7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.775222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 959/1898] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:12 2022] DELETE /identity/v3/users/4def03c2cc9d4c4184859d3dc2072ca7 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:12.866685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116d0dc088614961bcbc9b7dde44a6c7', 'request_id': 'req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.880610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116d0dc088614961bcbc9b7dde44a6c7', 'request_id': 'req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.893638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116d0dc088614961bcbc9b7dde44a6c7', 'request_id': 'req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.936962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] The token cache is being invalidate because project 116d0dc088614961bcbc9b7dde44a6c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:12.937405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.941471 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ca5d9ba-f8fe-4232-88fb-7dc8efed37da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:12.946712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 940/1899] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:12 2022] DELETE /identity/v3/projects/116d0dc088614961bcbc9b7dde44a6c7 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:12.952868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eed80-cff6-4f31-ac94-86a509ada991 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:12.953361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eed80-cff6-4f31-ac94-86a509ada991 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:12.953551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840eed80-cff6-4f31-ac94-86a509ada991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.026414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-840eed80-cff6-4f31-ac94-86a509ada991 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:13.031215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-840eed80-cff6-4f31-ac94-86a509ada991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:13.168090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 960/1900] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:13.175444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf251-9c6b-4371-9b02-9e884ddc53b6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.175905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf251-9c6b-4371-9b02-9e884ddc53b6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.175990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cf251-9c6b-4371-9b02-9e884ddc53b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.229572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-399cf251-9c6b-4371-9b02-9e884ddc53b6 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:13.231462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-399cf251-9c6b-4371-9b02-9e884ddc53b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:13.251697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 941/1901] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:13.255962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840eed80-cff6-4f31-ac94-86a509ada991 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.273593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840eed80-cff6-4f31-ac94-86a509ada991 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.290987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.290987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.290987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.291679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.292013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:13.294386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.298415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.326743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 961/1902] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:13.332903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cf251-9c6b-4371-9b02-9e884ddc53b6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.343632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cf251-9c6b-4371-9b02-9e884ddc53b6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.350086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.354798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.355549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.355689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.355820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.356677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.358943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.359079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:13.410136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048481db-07de-43ec-9609-38ee82b4e615 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.425246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 942/1903] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:13 2022] POST /identity/v3/projects => generated 379 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:13.426856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.518025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.520853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.521306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.522471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] PATH_INFO: `/v3/projects/8c4cc7988b994250a6875f037f32c897` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.528424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c4cc7988b994250a6875f037f32c897)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.528424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.540815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48f24b-bedf-4f2d-ad46-466cbb25c1de admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.551912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.552491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.552770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.553189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.553818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.555829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.602888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 943/1904] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:13 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:13.607242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.613182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:13.639721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8e712-d8f5-4276-ab40-832d0f691d2c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.673681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c4cc7988b994250a6875f037f32c897', 'request_id': 'req-f77c9064-68f4-445f-9ed8-1f37516ddbd0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:13.715057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c4cc7988b994250a6875f037f32c897', 'request_id': 'req-f77c9064-68f4-445f-9ed8-1f37516ddbd0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:13.740288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:13.742393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c4cc7988b994250a6875f037f32c897', 'request_id': 'req-f77c9064-68f4-445f-9ed8-1f37516ddbd0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:13.774825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] The token cache is being invalidate because project 8c4cc7988b994250a6875f037f32c897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:13.775705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:13.777090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:13.779873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 962/1905] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:13 2022] DELETE /identity/v3/projects/8c4cc7988b994250a6875f037f32c897 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:13.784953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.787400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:13.808531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.809627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.810236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.810794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.811768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.813682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.831592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 944/1906] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:13.837015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.839154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:13.854830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf436d-2eb9-45aa-9474-e5366f77ef2d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.861957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.862867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.863146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.863542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.864412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.866247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.880140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 945/1907] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:13.885211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:13.896451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d6556-7c4f-4824-b6c0-0d92e7953616 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.906166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f805ea8-230b-4af9-a358-95cca3d578d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.907829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f805ea8-230b-4af9-a358-95cca3d578d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.908449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f805ea8-230b-4af9-a358-95cca3d578d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.908979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f805ea8-230b-4af9-a358-95cca3d578d2 admin admin] PATH_INFO: `/v3/projects/5ba823ed1bac4e509331988a1609a822/users/916958fdf9574a17bc2fdaf05d487033/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.924265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77c9064-68f4-445f-9ed8-1f37516ddbd0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:13.931032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:13.932070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.932437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.933559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] PATH_INFO: `/v3/users/ca926268f2484010b1426f75bec6b552` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:13.935076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca926268f2484010b1426f75bec6b552)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.937574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.938104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f805ea8-230b-4af9-a358-95cca3d578d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba823ed1bac4e509331988a1609a822, user_id=916958fdf9574a17bc2fdaf05d487033, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:13.941452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f805ea8-230b-4af9-a358-95cca3d578d2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:13.977780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 946/1908] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:13 2022] PUT /identity/v3/projects/5ba823ed1bac4e509331988a1609a822/users/916958fdf9574a17bc2fdaf05d487033/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:13.981065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d5f7c-f7fe-48d0-9da3-976695805f6e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:13.981363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d5f7c-f7fe-48d0-9da3-976695805f6e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:13.982032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d5f7c-f7fe-48d0-9da3-976695805f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.018724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ad0d5f7c-f7fe-48d0-9da3-976695805f6e None None] MFA Rules not processed for user `916958fdf9574a17bc2fdaf05d487033`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:14.022077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ad0d5f7c-f7fe-48d0-9da3-976695805f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:14.037884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca926268f2484010b1426f75bec6b552'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:14.039422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca926268f2484010b1426f75bec6b552'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:14.040155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca926268f2484010b1426f75bec6b552'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:14.051274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca926268f2484010b1426f75bec6b552'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:14.095074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca926268f2484010b1426f75bec6b552'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:14.157675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 963/1909] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:13 2022] DELETE /identity/v3/users/ca926268f2484010b1426f75bec6b552 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:14.227715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 947/1910] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:14.232596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:14.234560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:14.295998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:14.396262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8160e-b38b-48ad-aa25-13c3f8ca7642 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.404455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.404710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.404896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.405164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.406462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.408656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.468744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 964/1911] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:14 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:14.472050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d5f7c-f7fe-48d0-9da3-976695805f6e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:14.672159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d5f7c-f7fe-48d0-9da3-976695805f6e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.738893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:14.830062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.832798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.833311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.836757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.838194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.843423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:14.903917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 948/1912] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:14 2022] POST /identity/v3/users => generated 297 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:14.913178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:14.949590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613810c-4282-466b-b765-9d20901173b6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:14.963695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:14.969222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:14.970034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:14.971239 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:14.974531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:14.979154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.008539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 965/1913] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:15.011825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.063007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a410f6-eb92-45ca-b581-c2535eaa1ad9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.094866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.100406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.100406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.100406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] PATH_INFO: `/v3/projects/f61ba96134314c3d947b8069eaf12ab0/users/8ec72dac0cbc42ac9c1360efa5994533/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.165410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61ba96134314c3d947b8069eaf12ab0, user_id=8ec72dac0cbc42ac9c1360efa5994533, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.172152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.237520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 949/1914] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:15 2022] PUT /identity/v3/projects/f61ba96134314c3d947b8069eaf12ab0/users/8ec72dac0cbc42ac9c1360efa5994533/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:15.242840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.261600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.267235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:15.276439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95b918-02af-4a7f-8d8c-fd4ecfd63359 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.396637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.398768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.399218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.399690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.401547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.404252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.420275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 966/1915] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:15.422822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.436541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d6354-d42d-4aed-b217-c3947f4c5ea7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.436798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5578b3-c29a-4e0e-ae8c-739466b4f935 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.441172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87419094-61d6-4593-8a84-9d4395d2058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.441620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.442299 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87419094-61d6-4593-8a84-9d4395d2058d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.442587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.442728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87419094-61d6-4593-8a84-9d4395d2058d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.442869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.443093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87419094-61d6-4593-8a84-9d4395d2058d admin admin] PATH_INFO: `/v3/projects/f61ba96134314c3d947b8069eaf12ab0/users/8ec72dac0cbc42ac9c1360efa5994533/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.443192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] PATH_INFO: `/v3/projects/d1f8ef3d1c144a6688c838c16e4a4892` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.445752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f8ef3d1c144a6688c838c16e4a4892)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.445933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87419094-61d6-4593-8a84-9d4395d2058d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61ba96134314c3d947b8069eaf12ab0, user_id=8ec72dac0cbc42ac9c1360efa5994533, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.447866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.449365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87419094-61d6-4593-8a84-9d4395d2058d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.464850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 967/1916] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:15 2022] PUT /identity/v3/projects/f61ba96134314c3d947b8069eaf12ab0/users/8ec72dac0cbc42ac9c1360efa5994533/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:15.476558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.476920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.477130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.512828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] MFA Rules not processed for user `8ec72dac0cbc42ac9c1360efa5994533`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:15.514270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:15.517248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f8ef3d1c144a6688c838c16e4a4892', 'request_id': 'req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:15.531234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f8ef3d1c144a6688c838c16e4a4892', 'request_id': 'req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:15.540858 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f8ef3d1c144a6688c838c16e4a4892', 'request_id': 'req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:15.573990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] The token cache is being invalidate because project d1f8ef3d1c144a6688c838c16e4a4892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:15.573990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:15.575556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cb315fd8-8730-4342-9dc0-f51cfe35b9e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:15.578038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 950/1917] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:15 2022] DELETE /identity/v3/projects/d1f8ef3d1c144a6688c838c16e4a4892 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:15.584397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ed44a-2df2-4d60-8883-69551741246f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.585653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ed44a-2df2-4d60-8883-69551741246f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.586312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ed44a-2df2-4d60-8883-69551741246f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.614390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-311ed44a-2df2-4d60-8883-69551741246f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:15.616252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-311ed44a-2df2-4d60-8883-69551741246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:15.628169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 968/1918] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:15.641074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.647371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:15.664368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:15.682713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:15.762544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cacd9-7dd0-4108-bfa5-db1eb5944a1f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.768829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.769830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.770302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.770785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.771051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 951/1919] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:15.780751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11d154-5e18-4204-b8d9-c49ac1d63f8d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.781045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11d154-5e18-4204-b8d9-c49ac1d63f8d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.781297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11d154-5e18-4204-b8d9-c49ac1d63f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.790834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.794087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.807631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 969/1920] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:15.810999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9e11d154-5e18-4204-b8d9-c49ac1d63f8d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:15.812484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9e11d154-5e18-4204-b8d9-c49ac1d63f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:15.828973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 952/1921] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:15.834632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.871891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cea90e-1242-4d5b-919c-60a766d418c4 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.877865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.879465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.879465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.879465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.880004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:15.880321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.882222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:15.904064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 970/1922] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:15.908271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11d154-5e18-4204-b8d9-c49ac1d63f8d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:15.926944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11d154-5e18-4204-b8d9-c49ac1d63f8d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:15.935274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:15.935974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:15.936258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:15.936523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:15.937219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:15.943486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:16.021059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 953/1923] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:15 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:16.026429 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:16.052257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c307d7-d1b2-422c-93cc-32a17c811d03 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:16.197974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:16.200024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:16.200371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:16.200657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:16.201369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:16.203366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:16.268877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 971/1924] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:16 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:16.273200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:16.297803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b2c22-0292-44b0-afc2-33c41318abf6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:16.312207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:16.312898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:16.313182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:16.313465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:16.314263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:16.319029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:16.353537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 954/1925] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:16.357836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:16.372359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480ff11-71c4-42b5-bca1-f12ff1dd45c9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:16.385877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115e2c6-ffc2-46d2-af2c-32688bebe154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:16.386678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115e2c6-ffc2-46d2-af2c-32688bebe154 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:16.386976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115e2c6-ffc2-46d2-af2c-32688bebe154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:16.389137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115e2c6-ffc2-46d2-af2c-32688bebe154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:16.392085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115e2c6-ffc2-46d2-af2c-32688bebe154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:16.397185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115e2c6-ffc2-46d2-af2c-32688bebe154 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:16.427103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 972/1926] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:16.437019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:16.456894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b54c72-2fb4-4e42-91dc-3028c9062711 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:16.472111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:16.475767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:16.476992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:16.478066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] PATH_INFO: `/v3/projects/9e222bd5af6e4f1c823147c870262708/users/47c5a97e2b2a462e92cadbaf8759cb77/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:16.533153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e222bd5af6e4f1c823147c870262708, user_id=47c5a97e2b2a462e92cadbaf8759cb77, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:16.538999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:16.585113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 955/1927] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:16 2022] PUT /identity/v3/projects/9e222bd5af6e4f1c823147c870262708/users/47c5a97e2b2a462e92cadbaf8759cb77/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:16.598867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:16.599161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:16.600647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:16.673153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] MFA Rules not processed for user `47c5a97e2b2a462e92cadbaf8759cb77`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:16.674433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:16.835762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 973/1928] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:16.848866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:16.852348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:16.951975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:16.956700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:16.969114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:16.980569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f18582-92c4-421c-b4f2-6e1c33a9b58d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:16.987167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eaa4d0-a0f9-41b1-832c-30a294e4741f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:16.988204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa4d0-a0f9-41b1-832c-30a294e4741f service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:16.988661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa4d0-a0f9-41b1-832c-30a294e4741f service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:16.991085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eaa4d0-a0f9-41b1-832c-30a294e4741f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.007572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaa4d0-a0f9-41b1-832c-30a294e4741f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:17.009555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eaa4d0-a0f9-41b1-832c-30a294e4741f service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:17.029234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 956/1929] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:17.065057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8d30d-36f2-4fbd-a636-79317e8df909 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:17.071928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:17.072953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:17.073306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:17.073762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] PATH_INFO: `/v3/users/916958fdf9574a17bc2fdaf05d487033` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.075338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916958fdf9574a17bc2fdaf05d487033)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:17.079052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:17.166018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916958fdf9574a17bc2fdaf05d487033'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:17.166924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916958fdf9574a17bc2fdaf05d487033'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:17.167686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916958fdf9574a17bc2fdaf05d487033'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:17.181549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916958fdf9574a17bc2fdaf05d487033'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:17.191852 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916958fdf9574a17bc2fdaf05d487033'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:17.232387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 974/1930] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:16 2022] DELETE /identity/v3/users/916958fdf9574a17bc2fdaf05d487033 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:17.236965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ef080-3f5c-4595-8773-b988911c14bc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:17.237766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ef080-3f5c-4595-8773-b988911c14bc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:17.237766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ef080-3f5c-4595-8773-b988911c14bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.275352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-421ef080-3f5c-4595-8773-b988911c14bc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:17.278979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-421ef080-3f5c-4595-8773-b988911c14bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:17.393704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:17.395957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:17.399797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 957/1931] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:17.496905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcbabf-aaa2-48e9-ac68-278d04932ab3 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:17.497200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcbabf-aaa2-48e9-ac68-278d04932ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:17.497526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcbabf-aaa2-48e9-ac68-278d04932ab3 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.500506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 958/1932] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:13:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:13:17.512294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:17.512554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:17.512727 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.521285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e864ca5-7240-4439-9d51-ee8498a41fe6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:17.523476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:17.533339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:17.534295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:17.536515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:17.536856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.549259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:17.552880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:17.559817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:17.561993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:17.601509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 975/1933] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:17.757643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 959/1934] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:13:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:17.870618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:17.872664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:17.906167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd51a13-6f87-40fd-8aed-4e8867f3abe5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:17.924511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:17.925617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:17.927380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:17.934319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:17.953517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:17.957123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:17.981212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 976/1935] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:18.395413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:18.398541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:18.420630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed598f-2bdb-4eae-aa86-e08e8033221b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:18.427749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:18.428998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:18.429289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:18.429573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] PATH_INFO: `/v3/projects/5ba823ed1bac4e509331988a1609a822` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:18.432206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba823ed1bac4e509331988a1609a822)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:18.434609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:18.526530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba823ed1bac4e509331988a1609a822', 'request_id': 'req-d9040496-1c15-4a10-a21a-1dff2ec070d4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.550359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba823ed1bac4e509331988a1609a822', 'request_id': 'req-d9040496-1c15-4a10-a21a-1dff2ec070d4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.567312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba823ed1bac4e509331988a1609a822', 'request_id': 'req-d9040496-1c15-4a10-a21a-1dff2ec070d4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.599949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] The token cache is being invalidate because project 5ba823ed1bac4e509331988a1609a822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:18.600236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.612047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.619690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:18.624626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 960/1936] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:18 2022] DELETE /identity/v3/projects/5ba823ed1bac4e509331988a1609a822 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:18.646940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e61f6-08e8-4ca8-967f-5f54446c1d17 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:18.674065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:18.789979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:18.790891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:18.791334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:18.791623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] PATH_INFO: `/v3/users/8ec72dac0cbc42ac9c1360efa5994533` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:18.792919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec72dac0cbc42ac9c1360efa5994533)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:18.794952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:18.883662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec72dac0cbc42ac9c1360efa5994533'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.885174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec72dac0cbc42ac9c1360efa5994533'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.886268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec72dac0cbc42ac9c1360efa5994533'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.904288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec72dac0cbc42ac9c1360efa5994533'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.921336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1b57a36-0c08-4853-8fba-d5a4fbad312e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec72dac0cbc42ac9c1360efa5994533'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:18.957769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 977/1937] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:18 2022] DELETE /identity/v3/users/8ec72dac0cbc42ac9c1360efa5994533 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:19.463745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:19.467124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:19.511335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040496-1c15-4a10-a21a-1dff2ec070d4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:19.651216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:19.652478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:19.652953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:19.653383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] PATH_INFO: `/v3/projects/f61ba96134314c3d947b8069eaf12ab0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:19.655265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61ba96134314c3d947b8069eaf12ab0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:19.660600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:19.738054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:19.815978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61ba96134314c3d947b8069eaf12ab0', 'request_id': 'req-81945112-ded9-4c85-b774-930f6e72ad4e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:19.843591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61ba96134314c3d947b8069eaf12ab0', 'request_id': 'req-81945112-ded9-4c85-b774-930f6e72ad4e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:19.870063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61ba96134314c3d947b8069eaf12ab0', 'request_id': 'req-81945112-ded9-4c85-b774-930f6e72ad4e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:19.924781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] The token cache is being invalidate because project f61ba96134314c3d947b8069eaf12ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:19.925671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:19.928889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81945112-ded9-4c85-b774-930f6e72ad4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:19.932776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 961/1938] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:19 2022] DELETE /identity/v3/projects/f61ba96134314c3d947b8069eaf12ab0 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:19.941336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce82c05-45af-4a66-a3fa-1c2b2c80742c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:19.942255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce82c05-45af-4a66-a3fa-1c2b2c80742c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:19.942407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce82c05-45af-4a66-a3fa-1c2b2c80742c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:19.988156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ce82c05-45af-4a66-a3fa-1c2b2c80742c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:19.990390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ce82c05-45af-4a66-a3fa-1c2b2c80742c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:20.200983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 978/1939] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:20.211038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2cf8-c120-41ff-a46f-9ebc5cdb71ff None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.211483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2cf8-c120-41ff-a46f-9ebc5cdb71ff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.214000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c2cf8-c120-41ff-a46f-9ebc5cdb71ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.240120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-517c2cf8-c120-41ff-a46f-9ebc5cdb71ff None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:20.241349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-517c2cf8-c120-41ff-a46f-9ebc5cdb71ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:20.255952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 962/1940] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:20.260329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce82c05-45af-4a66-a3fa-1c2b2c80742c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.271289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce82c05-45af-4a66-a3fa-1c2b2c80742c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.280627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.281098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.281379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.281664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.282447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:20.283142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.285203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.315625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 979/1941] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:13:20.317502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c2cf8-c120-41ff-a46f-9ebc5cdb71ff None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.331023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c2cf8-c120-41ff-a46f-9ebc5cdb71ff None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.336502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.337418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.337711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.338030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.338962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.341834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.401307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 963/1942] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:20 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:20.402860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.439695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37ce77-2b13-4c4c-96ed-91d39fac0b13 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.503368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:20.585499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.586996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.586996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.587467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.588102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.590850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.622122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 980/1943] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:20 2022] POST /identity/v3/users => generated 292 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:20.623361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.639777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922749b-b21d-42ff-9a56-b043f4cf9d17 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.649139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.649784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.650229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.650707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.652484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.655908 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.674656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 964/1944] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:20.694050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.709224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ee470-2859-4acc-8b89-4b167a65fb12 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.714985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306d6a6-dff9-4b2d-9e09-12bf28a2f040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.715796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306d6a6-dff9-4b2d-9e09-12bf28a2f040 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.716083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306d6a6-dff9-4b2d-9e09-12bf28a2f040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.716362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306d6a6-dff9-4b2d-9e09-12bf28a2f040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.717116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306d6a6-dff9-4b2d-9e09-12bf28a2f040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.718811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306d6a6-dff9-4b2d-9e09-12bf28a2f040 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.739624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 981/1945] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:20.746315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:20.763869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115e577-7626-4f20-ae16-f09ff45ad4a7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:20.772923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:20.774609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.775355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.775866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] PATH_INFO: `/v3/projects/3a5052f5bd2947a3abbde0ab4b3d6967/users/af6fd7c6f7184c488229b6b761400f5a/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.796256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5052f5bd2947a3abbde0ab4b3d6967, user_id=af6fd7c6f7184c488229b6b761400f5a, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:20.801833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:20.823380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 965/1946] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:20 2022] PUT /identity/v3/projects/3a5052f5bd2947a3abbde0ab4b3d6967/users/af6fd7c6f7184c488229b6b761400f5a/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:20.836564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c178a2-ee6f-4a03-a23c-c2a37ce99ce9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:20.837740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c178a2-ee6f-4a03-a23c-c2a37ce99ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:20.838253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c178a2-ee6f-4a03-a23c-c2a37ce99ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:20.878281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-74c178a2-ee6f-4a03-a23c-c2a37ce99ce9 None None] MFA Rules not processed for user `af6fd7c6f7184c488229b6b761400f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:20.880422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-74c178a2-ee6f-4a03-a23c-c2a37ce99ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:20.987241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 982/1947] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:20.995421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.128196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67304d-61ac-49c2-b1ec-8b9fa6b73606 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:21.151402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:21.152332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.152688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.153205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.154597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:21.163227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:21.244740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 966/1948] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:20 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:21.246072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c178a2-ee6f-4a03-a23c-c2a37ce99ce9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.273811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c178a2-ee6f-4a03-a23c-c2a37ce99ce9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:21.382895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:21.383638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.383957 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.384246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.385050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:21.387466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:21.430708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 983/1949] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:21 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:21.434183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.455833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd5757-a50f-4747-aa7d-5fdca48754bc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:21.488368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:21.489533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.490063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.490785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.492359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:21.504190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:21.527956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 967/1950] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:21.535583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.552156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1004cab7-7421-484e-b437-47006d035818 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:21.565156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:21.566505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.567062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.567248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] PATH_INFO: `/v3/projects/c7faa808d4f4484892dcecd178bf92be/users/5603449b3f6f46409c26f911ad76d30f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.599736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7faa808d4f4484892dcecd178bf92be, user_id=5603449b3f6f46409c26f911ad76d30f, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:21.602520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:21.622066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:21.634195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 984/1951] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:21 2022] PUT /identity/v3/projects/c7faa808d4f4484892dcecd178bf92be/users/5603449b3f6f46409c26f911ad76d30f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:21.640188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.655085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae810b0-6b2d-4e5c-9294-1db8538eb613 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:21.683118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:21.738450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879104d-43b3-4681-bbf9-4897b4923142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:21.739725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879104d-43b3-4681-bbf9-4897b4923142 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.740841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879104d-43b3-4681-bbf9-4897b4923142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.741662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879104d-43b3-4681-bbf9-4897b4923142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.744384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879104d-43b3-4681-bbf9-4897b4923142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:21.748389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879104d-43b3-4681-bbf9-4897b4923142 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:21.773354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 968/1952] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:21.777963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:21.790553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fa667-90e4-4e8f-91a2-cfc241c116f3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:21.796329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:21.797500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.798117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.798481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] PATH_INFO: `/v3/projects/c7faa808d4f4484892dcecd178bf92be/users/5603449b3f6f46409c26f911ad76d30f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.800850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7faa808d4f4484892dcecd178bf92be, user_id=5603449b3f6f46409c26f911ad76d30f, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:21.805452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:21.833358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 985/1953] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:21 2022] PUT /identity/v3/projects/c7faa808d4f4484892dcecd178bf92be/users/5603449b3f6f46409c26f911ad76d30f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:21.835513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:21.836277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:21.837016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:21.888301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] MFA Rules not processed for user `5603449b3f6f46409c26f911ad76d30f`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:21.889608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:22.053102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 969/1954] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:22.066467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:22.068478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:22.186948 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04301fa-c9df-4568-a57a-6c7b3ade965a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:22.199627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:22.200699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:22.201219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:22.201839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:22.330418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:22.333358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:22.346842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 986/1955] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:13:32.043786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=54 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:32.341152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:33.556518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:33.559248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:33.670220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e11aaa-9070-4275-95c8-eb238007f35a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:33.677025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:33.679537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:33.679946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:33.680330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] PATH_INFO: `/v3/users/bae9ecea988c4a13ad38fc3312eedbc9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:33.682739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bae9ecea988c4a13ad38fc3312eedbc9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:33.685243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:33.774660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bae9ecea988c4a13ad38fc3312eedbc9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:33.775522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bae9ecea988c4a13ad38fc3312eedbc9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:33.787099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bae9ecea988c4a13ad38fc3312eedbc9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:33.788315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bae9ecea988c4a13ad38fc3312eedbc9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:33.803690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bae9ecea988c4a13ad38fc3312eedbc9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:33.827804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 970/1956] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:33 2022] DELETE /identity/v3/users/bae9ecea988c4a13ad38fc3312eedbc9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:34.039722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:34.053643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bd688-8fcb-4f18-9444-0557997dbfdd service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:34.123762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:34.124690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:34.125070 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:34.125363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] PATH_INFO: `/v3/projects/dac369a021ea484fb12cc8e8f586ab54` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:34.126946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac369a021ea484fb12cc8e8f586ab54)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:34.128874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:34.186461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac369a021ea484fb12cc8e8f586ab54', 'request_id': 'req-b95a4d78-2740-4858-b767-bc4d0126161b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.196176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac369a021ea484fb12cc8e8f586ab54', 'request_id': 'req-b95a4d78-2740-4858-b767-bc4d0126161b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.205748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac369a021ea484fb12cc8e8f586ab54', 'request_id': 'req-b95a4d78-2740-4858-b767-bc4d0126161b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.220196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] The token cache is being invalidate because project dac369a021ea484fb12cc8e8f586ab54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:34.220546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.222681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.224444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 987/1957] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:34 2022] DELETE /identity/v3/projects/dac369a021ea484fb12cc8e8f586ab54 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:34.229223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:34.231755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:34.246248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87f12-5a82-4572-8555-d0ae92ea7748 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:34.311433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:34.312454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:34.312756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:34.313046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] PATH_INFO: `/v3/users/1c25775db8e44a2ba8523597461106ff` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:34.314296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c25775db8e44a2ba8523597461106ff)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:34.316263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:34.341820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:34.370718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c25775db8e44a2ba8523597461106ff'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.371425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c25775db8e44a2ba8523597461106ff'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.378441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c25775db8e44a2ba8523597461106ff'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.379184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c25775db8e44a2ba8523597461106ff'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.386805 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b1248982-9aa1-468e-a63e-92365622fb2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c25775db8e44a2ba8523597461106ff'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:34.405823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 971/1958] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:34 2022] DELETE /identity/v3/users/1c25775db8e44a2ba8523597461106ff => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:34.844247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:34.845846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:34.859675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a4d78-2740-4858-b767-bc4d0126161b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:34.956600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:34.957391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:34.957678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:34.958103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] PATH_INFO: `/v3/projects/4ac04c61743b408ba01b92a225fdf1c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:34.959336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac04c61743b408ba01b92a225fdf1c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:34.961302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.020323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:35.048878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac04c61743b408ba01b92a225fdf1c8', 'request_id': 'req-74dc963a-1e54-4630-ba13-44e17a2c88f0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.058464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac04c61743b408ba01b92a225fdf1c8', 'request_id': 'req-74dc963a-1e54-4630-ba13-44e17a2c88f0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.066520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac04c61743b408ba01b92a225fdf1c8', 'request_id': 'req-74dc963a-1e54-4630-ba13-44e17a2c88f0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.083401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] The token cache is being invalidate because project 4ac04c61743b408ba01b92a225fdf1c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:35.083401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.084359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74dc963a-1e54-4630-ba13-44e17a2c88f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:35.086109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 988/1959] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:34 2022] DELETE /identity/v3/projects/4ac04c61743b408ba01b92a225fdf1c8 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:35.091740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ae22a-49cb-4909-b291-cf41e509c9c1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.092327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ae22a-49cb-4909-b291-cf41e509c9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.092623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ae22a-49cb-4909-b291-cf41e509c9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.115476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-946ae22a-49cb-4909-b291-cf41e509c9c1 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:35.116986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-946ae22a-49cb-4909-b291-cf41e509c9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:35.191355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 972/1960] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:35.200325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0a78-5e8f-4a54-9c7b-f5c15eabc523 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.200683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0a78-5e8f-4a54-9c7b-f5c15eabc523 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.200891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e0a78-5e8f-4a54-9c7b-f5c15eabc523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.224591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-220e0a78-5e8f-4a54-9c7b-f5c15eabc523 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:35.226193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-220e0a78-5e8f-4a54-9c7b-f5c15eabc523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:35.238854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 989/1961] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:35.243993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ae22a-49cb-4909-b291-cf41e509c9c1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.259077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ae22a-49cb-4909-b291-cf41e509c9c1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.264829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.265575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.265917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.266203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.267050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:35.267356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.269086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.285337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 973/1962] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:13:35.290722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e0a78-5e8f-4a54-9c7b-f5c15eabc523 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.307167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e0a78-5e8f-4a54-9c7b-f5c15eabc523 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.312822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.313509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.313832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.315405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.315405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.317467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.355462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 990/1963] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:35 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:35.361974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.375675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03963e5e-b931-405c-861e-377b339ab4c4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.445837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.446531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.446817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.447100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.447800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.450398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.472678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 974/1964] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:35 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:35.478686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.493923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc448-8ee9-4990-ad74-647c8478e6d4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.502156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.503312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.503805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.504292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.505587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.508004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.520607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 991/1965] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:35.525104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.536845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1cb63-116d-4c1a-89f9-fce498d1abfe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.542513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d47cf-b2be-44da-8bf8-9c8b40f3f221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.543091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d47cf-b2be-44da-8bf8-9c8b40f3f221 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.543388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d47cf-b2be-44da-8bf8-9c8b40f3f221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.543670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d47cf-b2be-44da-8bf8-9c8b40f3f221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.544401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d47cf-b2be-44da-8bf8-9c8b40f3f221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.546319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d47cf-b2be-44da-8bf8-9c8b40f3f221 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.560236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 975/1966] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:35.566112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.577399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba0a8-224c-4c32-8749-9a27ce46528b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.585979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.587327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.587612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.587967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] PATH_INFO: `/v3/projects/7f8275641347464191d00af2779bd73f/users/68a545efaafb45c19bb45bbd6a80717c/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.604211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8275641347464191d00af2779bd73f, user_id=68a545efaafb45c19bb45bbd6a80717c, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.608826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.624886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 992/1967] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:35 2022] PUT /identity/v3/projects/7f8275641347464191d00af2779bd73f/users/68a545efaafb45c19bb45bbd6a80717c/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:35.636640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2b85-7edb-4931-8731-a7294311677e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.637041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2b85-7edb-4931-8731-a7294311677e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.637308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2b85-7edb-4931-8731-a7294311677e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.647042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:35.671146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8ccd2b85-7edb-4931-8731-a7294311677e None None] MFA Rules not processed for user `68a545efaafb45c19bb45bbd6a80717c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:35.672379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8ccd2b85-7edb-4931-8731-a7294311677e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:35.750902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 976/1968] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:35.762378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:35.764460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:35.829527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b18f43-6846-40d0-84c8-ab65cbbc83cb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:35.834560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:35.835621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:35.836275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:35.836598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:35.847808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:35.850288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:35.861786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 993/1969] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:36.999292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd2b85-7edb-4931-8731-a7294311677e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:37.011752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd2b85-7edb-4931-8731-a7294311677e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:37.015487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204dcc-b986-4623-b944-20485ebcb2b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:37.016151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204dcc-b986-4623-b944-20485ebcb2b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:37.016438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204dcc-b986-4623-b944-20485ebcb2b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:37.016720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204dcc-b986-4623-b944-20485ebcb2b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:37.018601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-95204dcc-b986-4623-b944-20485ebcb2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:37.126660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204dcc-b986-4623-b944-20485ebcb2b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:37.128131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95204dcc-b986-4623-b944-20485ebcb2b2 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:37.149380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 977/1970] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:39.509617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:39.576907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9cce4-aac8-4b4f-93f8-1730c3a856b9 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:39.581856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:39.582730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:39.583015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:39.583293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] PATH_INFO: `/v3/users/68a545efaafb45c19bb45bbd6a80717c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:39.584523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a545efaafb45c19bb45bbd6a80717c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:39.586567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:39.588368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:39.641403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a545efaafb45c19bb45bbd6a80717c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.642421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a545efaafb45c19bb45bbd6a80717c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.643037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a545efaafb45c19bb45bbd6a80717c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.656290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a545efaafb45c19bb45bbd6a80717c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.663970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a545efaafb45c19bb45bbd6a80717c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:39.682745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 994/1971] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:39 2022] DELETE /identity/v3/users/68a545efaafb45c19bb45bbd6a80717c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:39.687743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:39.688304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:39.688716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:39.711027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:39.712580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:39.800733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 978/1972] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:39.811355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:39.813253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:39.883313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435faa8-324d-4fad-a796-002f26633ebe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:39.888759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc542277-7b63-4802-bdd3-782645b69bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:39.889809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542277-7b63-4802-bdd3-782645b69bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:39.890105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542277-7b63-4802-bdd3-782645b69bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:39.890513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc542277-7b63-4802-bdd3-782645b69bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:39.902947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc542277-7b63-4802-bdd3-782645b69bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:39.904994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc542277-7b63-4802-bdd3-782645b69bb6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:39.916356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 995/1973] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:40.246943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.248604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:40.272020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98cdc-9c5e-41c8-9df1-55592ea1cbfa None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.286258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.287533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.287987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.288373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] PATH_INFO: `/v3/projects/7f8275641347464191d00af2779bd73f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.291460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f8275641347464191d00af2779bd73f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.294738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.351535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:40.394054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f8275641347464191d00af2779bd73f', 'request_id': 'req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.404228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f8275641347464191d00af2779bd73f', 'request_id': 'req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.415060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f8275641347464191d00af2779bd73f', 'request_id': 'req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.430896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] The token cache is being invalidate because project 7f8275641347464191d00af2779bd73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:40.431190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.431859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5f8ae8d6-fdd4-4d98-a330-6fcfa2656bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:40.433498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 979/1974] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:40 2022] DELETE /identity/v3/projects/7f8275641347464191d00af2779bd73f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:40.438523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ae02e-e87f-4077-9f6d-6b7b9bd1811a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.438817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ae02e-e87f-4077-9f6d-6b7b9bd1811a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.439101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ae02e-e87f-4077-9f6d-6b7b9bd1811a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.463157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-569ae02e-e87f-4077-9f6d-6b7b9bd1811a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:40.464355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-569ae02e-e87f-4077-9f6d-6b7b9bd1811a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:40.566293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e2e40-f624-4f52-b1fb-2a3de4118608 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.566677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e2e40-f624-4f52-b1fb-2a3de4118608 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.566896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e2e40-f624-4f52-b1fb-2a3de4118608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.576742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 996/1975] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:40.593351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-468e2e40-f624-4f52-b1fb-2a3de4118608 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:40.595523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-468e2e40-f624-4f52-b1fb-2a3de4118608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:40.611718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 980/1976] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:40.613076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ae02e-e87f-4077-9f6d-6b7b9bd1811a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.627310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ae02e-e87f-4077-9f6d-6b7b9bd1811a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.637021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.637769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.638168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.638504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.639543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:40.639859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.641635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.659147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 997/1977] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:40.663949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e2e40-f624-4f52-b1fb-2a3de4118608 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.675817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e2e40-f624-4f52-b1fb-2a3de4118608 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.685765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.687070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.687847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.688419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.689673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.693395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.734839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 981/1978] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:40 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:40.736210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.748391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1f4d-6fae-4512-975b-acfcdda8b45e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.754158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:40.827869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.828664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.829184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.829535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.830355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.832417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.864885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 998/1979] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:40 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:40.865098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.881939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91deaca-3b0d-4087-b579-c1e031bb29b6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.886955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.887635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.887951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.888424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.889164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.890836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.910069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 982/1980] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:40.913785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.925424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc51f15-e10f-4f0c-8439-ff399d3f38a2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.930406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b830f05-5b94-4800-8773-19eb0c92d8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.931085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830f05-5b94-4800-8773-19eb0c92d8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.931376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830f05-5b94-4800-8773-19eb0c92d8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.931657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b830f05-5b94-4800-8773-19eb0c92d8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.932921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b830f05-5b94-4800-8773-19eb0c92d8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.934588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b830f05-5b94-4800-8773-19eb0c92d8c4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:40.952647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 999/1981] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:40.954160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:40.966068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f7f72-d669-48f7-a749-c12c27addce8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:40.971630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:40.972604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:40.972959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:40.973335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] PATH_INFO: `/v3/projects/4c53ab1ccd58493487638730786d0a03/users/527cff7026e24b2a8fd2aa78fdb74994/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:40.993567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c53ab1ccd58493487638730786d0a03, user_id=527cff7026e24b2a8fd2aa78fdb74994, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:40.996712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.013135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 983/1982] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:40 2022] PUT /identity/v3/projects/4c53ab1ccd58493487638730786d0a03/users/527cff7026e24b2a8fd2aa78fdb74994/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:41.024306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8572f-b318-4d1b-8641-f9736d85219f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.024599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8572f-b318-4d1b-8641-f9736d85219f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.024885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8572f-b318-4d1b-8641-f9736d85219f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.066024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9fc8572f-b318-4d1b-8641-f9736d85219f None None] MFA Rules not processed for user `527cff7026e24b2a8fd2aa78fdb74994`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:41.066024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9fc8572f-b318-4d1b-8641-f9736d85219f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.131637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1000/1983] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:41.147024 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.148792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.210114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519079-a506-4fea-90e9-3fa66a725d2c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.214806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34acade6-c6d4-418b-82c7-ea474be0c384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.215477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acade6-c6d4-418b-82c7-ea474be0c384 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.215879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acade6-c6d4-418b-82c7-ea474be0c384 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.216393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acade6-c6d4-418b-82c7-ea474be0c384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.230766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acade6-c6d4-418b-82c7-ea474be0c384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.232541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34acade6-c6d4-418b-82c7-ea474be0c384 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.242398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 984/1984] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:41.271792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8572f-b318-4d1b-8641-f9736d85219f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.363499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8572f-b318-4d1b-8641-f9736d85219f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.375925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.378083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.378083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.378083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] PATH_INFO: `/v3/users/527cff7026e24b2a8fd2aa78fdb74994` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.382893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=527cff7026e24b2a8fd2aa78fdb74994)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.385886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.452315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:41.510417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '527cff7026e24b2a8fd2aa78fdb74994'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.511896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '527cff7026e24b2a8fd2aa78fdb74994'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.512849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '527cff7026e24b2a8fd2aa78fdb74994'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.536941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '527cff7026e24b2a8fd2aa78fdb74994'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.556274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '527cff7026e24b2a8fd2aa78fdb74994'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:41.578721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1001/1985] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:41 2022] DELETE /identity/v3/users/527cff7026e24b2a8fd2aa78fdb74994 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:41.583625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.584086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.584376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.612475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:41.614017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.692313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:41.724293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 985/1986] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:41.732289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:41.734351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:41.820034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d000ab-21fb-4533-987a-617050a66f31 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:41.824577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b789703-b9f3-48e7-ae59-49b078bdd04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:41.825244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789703-b9f3-48e7-ae59-49b078bdd04d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:41.825526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789703-b9f3-48e7-ae59-49b078bdd04d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:41.825840 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b789703-b9f3-48e7-ae59-49b078bdd04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:41.836960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b789703-b9f3-48e7-ae59-49b078bdd04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:41.838457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b789703-b9f3-48e7-ae59-49b078bdd04d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:41.852529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1002/1987] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:42.197516 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.199455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.215245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde4510-2d81-4391-a73b-bbe3fd51514f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.220673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.223011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.223395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.223687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] PATH_INFO: `/v3/projects/4c53ab1ccd58493487638730786d0a03` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.224945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c53ab1ccd58493487638730786d0a03)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.227080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.293009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c53ab1ccd58493487638730786d0a03', 'request_id': 'req-50b78f73-06d8-4b47-8dcc-dde682f01b4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.301287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c53ab1ccd58493487638730786d0a03', 'request_id': 'req-50b78f73-06d8-4b47-8dcc-dde682f01b4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.308705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c53ab1ccd58493487638730786d0a03', 'request_id': 'req-50b78f73-06d8-4b47-8dcc-dde682f01b4d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.319875 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] The token cache is being invalidate because project 4c53ab1ccd58493487638730786d0a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:42.320165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.320822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-50b78f73-06d8-4b47-8dcc-dde682f01b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.322437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 986/1988] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:42 2022] DELETE /identity/v3/projects/4c53ab1ccd58493487638730786d0a03 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:42.328194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdb82e-22e9-406a-a782-792aed7725f5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.328794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdb82e-22e9-406a-a782-792aed7725f5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.329079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdb82e-22e9-406a-a782-792aed7725f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.349089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ebfdb82e-22e9-406a-a782-792aed7725f5 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:42.350363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ebfdb82e-22e9-406a-a782-792aed7725f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.419785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1003/1989] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:42.428603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5243f-6323-4d2a-91a2-2980481c90f2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.429318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5243f-6323-4d2a-91a2-2980481c90f2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.429694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5243f-6323-4d2a-91a2-2980481c90f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.457496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7ee5243f-6323-4d2a-91a2-2980481c90f2 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:42.458992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7ee5243f-6323-4d2a-91a2-2980481c90f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.477512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 987/1990] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:42.477658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdb82e-22e9-406a-a782-792aed7725f5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.483572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdb82e-22e9-406a-a782-792aed7725f5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.488661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.489334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.489621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.489937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.490850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:42.491164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.492777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.505385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1004/1991] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:42.510382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5243f-6323-4d2a-91a2-2980481c90f2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.520554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5243f-6323-4d2a-91a2-2980481c90f2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.527637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.528556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.528873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.529259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.530087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.532137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.563865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 988/1992] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:42.568188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.578774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05037185-6377-4597-bd12-74f6fb4b0a73 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.610122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:42.657448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.658169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.658459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.658741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.659434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.661369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.686959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1005/1993] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:42.688339 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.700604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b0992-a008-4149-9288-c7fe95828d9d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.705333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.707112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.707112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.707112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.707402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.708993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.720365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 989/1994] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:42.724583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.734858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b5205-da0a-434c-9952-bba73197aa4a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.739573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.740248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.740536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.740874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.741672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.743327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.753849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1006/1995] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:42.758741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.771654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78657aa-e406-424a-9c03-77bb9c416177 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.777211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9225c-3f20-4905-90bc-c7d8633f493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.778214 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9225c-3f20-4905-90bc-c7d8633f493f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.778623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9225c-3f20-4905-90bc-c7d8633f493f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.778923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9225c-3f20-4905-90bc-c7d8633f493f admin admin] PATH_INFO: `/v3/projects/f5e363bcf5ed47529b6464cf954a31ea/users/1a98ea9b90174a24b57b3875e9df2ecd/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.786904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:42.794766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.800710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9225c-3f20-4905-90bc-c7d8633f493f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e363bcf5ed47529b6464cf954a31ea, user_id=1a98ea9b90174a24b57b3875e9df2ecd, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.803989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9225c-3f20-4905-90bc-c7d8633f493f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.809179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbf914-69d3-46d5-ae30-f3867c05ee51 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:42.817353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:42.818728 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.819011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.819292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] PATH_INFO: `/v3/users/af6fd7c6f7184c488229b6b761400f5a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.822503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6fd7c6f7184c488229b6b761400f5a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:42.825030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:42.848500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 990/1996] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:42 2022] PUT /identity/v3/projects/f5e363bcf5ed47529b6464cf954a31ea/users/1a98ea9b90174a24b57b3875e9df2ecd/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:42.857310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52c872-bffa-493c-896d-a780988a72ae None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:42.860131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52c872-bffa-493c-896d-a780988a72ae None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:42.860589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52c872-bffa-493c-896d-a780988a72ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:42.900898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-db52c872-bffa-493c-896d-a780988a72ae None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:42.916117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-db52c872-bffa-493c-896d-a780988a72ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:42.933037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6fd7c6f7184c488229b6b761400f5a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.935435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6fd7c6f7184c488229b6b761400f5a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.936301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6fd7c6f7184c488229b6b761400f5a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.956423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6fd7c6f7184c488229b6b761400f5a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:42.968617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-dc4f257b-b585-467d-8879-c3f1617d6072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6fd7c6f7184c488229b6b761400f5a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:43.006415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1007/1997] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:42 2022] DELETE /identity/v3/users/af6fd7c6f7184c488229b6b761400f5a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:43.011809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416cf71-eba2-4c5e-8cfe-bdd17737dcf7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.012225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416cf71-eba2-4c5e-8cfe-bdd17737dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.012663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416cf71-eba2-4c5e-8cfe-bdd17737dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.043531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5416cf71-eba2-4c5e-8cfe-bdd17737dcf7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:43.045149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5416cf71-eba2-4c5e-8cfe-bdd17737dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.123754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 991/1998] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:43.153647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1008/1999] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:43.158811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52c872-bffa-493c-896d-a780988a72ae None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.162052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-db52c872-bffa-493c-896d-a780988a72ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:43.178814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:43.258548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52c872-bffa-493c-896d-a780988a72ae None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.282291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416cf71-eba2-4c5e-8cfe-bdd17737dcf7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:43.320754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:43.342066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.342743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.343034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.343317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.362811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.364320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.375762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 992/2000] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:43.392933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416cf71-eba2-4c5e-8cfe-bdd17737dcf7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:43.404756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:43.405465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:43.405778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:43.406114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:43.421507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:43.423222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:43.437886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1009/2001] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:44.105063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.106474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:44.121389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf645-f3af-424c-9e03-172ea13ba13d service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.135970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.135970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.135970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.135970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] PATH_INFO: `/v3/projects/3a5052f5bd2947a3abbde0ab4b3d6967` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.136399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5052f5bd2947a3abbde0ab4b3d6967)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.136660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.197105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5052f5bd2947a3abbde0ab4b3d6967', 'request_id': 'req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.206996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5052f5bd2947a3abbde0ab4b3d6967', 'request_id': 'req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.214188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5052f5bd2947a3abbde0ab4b3d6967', 'request_id': 'req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.227417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] The token cache is being invalidate because project 3a5052f5bd2947a3abbde0ab4b3d6967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:44.227804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.228639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.230987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 993/2002] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:44 2022] DELETE /identity/v3/projects/3a5052f5bd2947a3abbde0ab4b3d6967 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:44.234949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.244574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f8273-3a8d-4ae5-8603-3c5590687d47 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:44.304910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:44.305863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:44.306147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:44.306424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] PATH_INFO: `/v3/users/5603449b3f6f46409c26f911ad76d30f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:44.307731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5603449b3f6f46409c26f911ad76d30f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:44.309683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:44.365622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5603449b3f6f46409c26f911ad76d30f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.366365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5603449b3f6f46409c26f911ad76d30f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.367012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5603449b3f6f46409c26f911ad76d30f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.377171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5603449b3f6f46409c26f911ad76d30f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.386675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-445cbf49-238f-4b2a-ac40-93b64338d010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5603449b3f6f46409c26f911ad76d30f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:44.404963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1010/2003] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:44 2022] DELETE /identity/v3/users/5603449b3f6f46409c26f911ad76d30f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:44.960177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:44.961629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:44.988133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2d889-50a9-4e97-b7dc-1dfa7c400ccd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.067735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:45.087236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.089114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.089559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.090030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] PATH_INFO: `/v3/projects/c7faa808d4f4484892dcecd178bf92be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.091455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7faa808d4f4484892dcecd178bf92be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.093531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.166813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7faa808d4f4484892dcecd178bf92be', 'request_id': 'req-86305115-6381-4c3c-a454-bb4f5222fe8b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:45.176141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7faa808d4f4484892dcecd178bf92be', 'request_id': 'req-86305115-6381-4c3c-a454-bb4f5222fe8b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:45.184238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7faa808d4f4484892dcecd178bf92be', 'request_id': 'req-86305115-6381-4c3c-a454-bb4f5222fe8b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:45.196373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] The token cache is being invalidate because project c7faa808d4f4484892dcecd178bf92be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:45.196797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:45.197572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-86305115-6381-4c3c-a454-bb4f5222fe8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:45.200254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 994/2004] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:44 2022] DELETE /identity/v3/projects/c7faa808d4f4484892dcecd178bf92be => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:45.208350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60233e-7535-4cb3-aa0a-c66aa763b5f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.208815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60233e-7535-4cb3-aa0a-c66aa763b5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.209428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60233e-7535-4cb3-aa0a-c66aa763b5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.263232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2f60233e-7535-4cb3-aa0a-c66aa763b5f2 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:45.265078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2f60233e-7535-4cb3-aa0a-c66aa763b5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:45.300111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.300512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.300922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.311485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:45.350411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1011/2005] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:45.360077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d89277-5624-4e88-9d1c-3e3d990f84ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.360996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d89277-5624-4e88-9d1c-3e3d990f84ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.361278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d89277-5624-4e88-9d1c-3e3d990f84ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.362289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] MFA Rules not processed for user `1a98ea9b90174a24b57b3875e9df2ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:45.363625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:45.383095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-05d89277-5624-4e88-9d1c-3e3d990f84ab None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:45.384325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-05d89277-5624-4e88-9d1c-3e3d990f84ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:45.397866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1012/2006] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:45.402868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d89277-5624-4e88-9d1c-3e3d990f84ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.415464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d89277-5624-4e88-9d1c-3e3d990f84ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.421199 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.422295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.422738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.423179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.424380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:45.424873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.427882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.443409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 995/2007] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:45.452804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.453929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1013/2008] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:13:45.456689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:45.458019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.468943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb110c2-9471-4365-b847-649550de34f6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.481475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.482358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.482651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.482933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.483647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.485786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.527298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1014/2009] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:45 2022] POST /identity/v3/projects => generated 359 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:45.528411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.575621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3997141-fd9c-43a4-b265-1b7fb2da4382 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.607148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:45.624382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34924da4-e00d-4efe-ba9b-8acd1573e25d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.629993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.630779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.631136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.631612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.632262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.634312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.637163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.638046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.638396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.638757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.662047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1015/2010] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:45 2022] POST /identity/v3/users => generated 280 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:45.663397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.677581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb413f5c-cb5e-4df0-a316-beef9e548205 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.686178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.686988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.687223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.687507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.688246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.690565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.706958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1016/2011] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:13:45.713657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.717312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.719031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.727213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543f6fa-46d9-46e9-91c1-b28ce480c6f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.730358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 996/2012] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:45.734122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0b10-388f-4fe0-87cb-bc8cd246d236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.734806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0b10-388f-4fe0-87cb-bc8cd246d236 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.735098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0b10-388f-4fe0-87cb-bc8cd246d236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.735383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0b10-388f-4fe0-87cb-bc8cd246d236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.736148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0b10-388f-4fe0-87cb-bc8cd246d236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.737872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0b10-388f-4fe0-87cb-bc8cd246d236 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.753103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1017/2013] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:45.757581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.769370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b3063-abfb-4e3b-a4f3-cd8a3ec32152 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:45.777013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:45.777953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.778246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.778607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] PATH_INFO: `/v3/projects/d91a5fc891c943828d22f6c60ed17bc5/users/f7e591cc7ea3434f80f87d31c70ed521/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.798367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91a5fc891c943828d22f6c60ed17bc5, user_id=f7e591cc7ea3434f80f87d31c70ed521, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:45.801455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:45.820473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 997/2014] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:45 2022] PUT /identity/v3/projects/d91a5fc891c943828d22f6c60ed17bc5/users/f7e591cc7ea3434f80f87d31c70ed521/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:45.832054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b14b4-3699-4a21-a653-f7965a7545b0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:45.832342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b14b4-3699-4a21-a653-f7965a7545b0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:45.832620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b14b4-3699-4a21-a653-f7965a7545b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:45.863023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e52b14b4-3699-4a21-a653-f7965a7545b0 None None] MFA Rules not processed for user `f7e591cc7ea3434f80f87d31c70ed521`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:45.864768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e52b14b4-3699-4a21-a653-f7965a7545b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:45.940793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:45.958086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1018/2015] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:45.994344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:45.996580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:46.110817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469be42a-c8f5-4b14-a91c-4c867880b433 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:46.120298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c5e44-b90e-4377-88d0-2cd83fae4d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:46.121150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c5e44-b90e-4377-88d0-2cd83fae4d47 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:46.121877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c5e44-b90e-4377-88d0-2cd83fae4d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:46.122404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c5e44-b90e-4377-88d0-2cd83fae4d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:46.144879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c5e44-b90e-4377-88d0-2cd83fae4d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:46.146579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c5e44-b90e-4377-88d0-2cd83fae4d47 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:46.162058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 998/2016] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:46.187155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b14b4-3699-4a21-a653-f7965a7545b0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:46.265274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b14b4-3699-4a21-a653-f7965a7545b0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:46.269898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:46.270777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:46.271057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:46.271330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] PATH_INFO: `/v3/users/f7e591cc7ea3434f80f87d31c70ed521` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:46.272515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e591cc7ea3434f80f87d31c70ed521)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:46.274490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:46.355636 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e591cc7ea3434f80f87d31c70ed521'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:46.356370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e591cc7ea3434f80f87d31c70ed521'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:46.357202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e591cc7ea3434f80f87d31c70ed521'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:46.367562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e591cc7ea3434f80f87d31c70ed521'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:46.378123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e591cc7ea3434f80f87d31c70ed521'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:46.402345 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1019/2017] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:46 2022] DELETE /identity/v3/users/f7e591cc7ea3434f80f87d31c70ed521 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:46.407030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:46.407505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:46.407612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:46.430667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:46.432052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:46.535779 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 999/2018] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:46.546415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:46.547874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:46.610920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf595c-7e52-4ff1-b429-a2c1c76392c5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:46.614757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b459c30c-2792-4792-b42b-ea9aa3d62c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:46.615534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459c30c-2792-4792-b42b-ea9aa3d62c8b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:46.616005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459c30c-2792-4792-b42b-ea9aa3d62c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:46.616293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b459c30c-2792-4792-b42b-ea9aa3d62c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:46.626011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459c30c-2792-4792-b42b-ea9aa3d62c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:46.627382 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b459c30c-2792-4792-b42b-ea9aa3d62c8b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:46.637595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1020/2019] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:46.939226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:46.940730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:46.948674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:46.955324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e51f13-ea5a-4f8a-b656-0493bd1aa418 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:46.962321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:46.963311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:46.963619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:46.963904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] PATH_INFO: `/v3/projects/d91a5fc891c943828d22f6c60ed17bc5` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:46.965273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91a5fc891c943828d22f6c60ed17bc5)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:46.967951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.018470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91a5fc891c943828d22f6c60ed17bc5', 'request_id': 'req-c005b719-e024-424d-8a79-cc21eba5fad8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:47.026629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91a5fc891c943828d22f6c60ed17bc5', 'request_id': 'req-c005b719-e024-424d-8a79-cc21eba5fad8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:47.032922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91a5fc891c943828d22f6c60ed17bc5', 'request_id': 'req-c005b719-e024-424d-8a79-cc21eba5fad8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:47.043266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] The token cache is being invalidate because project d91a5fc891c943828d22f6c60ed17bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:47.043548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:47.044193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c005b719-e024-424d-8a79-cc21eba5fad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:47.045771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1000/2020] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:46 2022] DELETE /identity/v3/projects/d91a5fc891c943828d22f6c60ed17bc5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:47.051366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c757e-c731-4d50-8b3f-c700b31104e9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.051693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c757e-c731-4d50-8b3f-c700b31104e9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.052001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c757e-c731-4d50-8b3f-c700b31104e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.072117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d5c757e-c731-4d50-8b3f-c700b31104e9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:47.073887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d5c757e-c731-4d50-8b3f-c700b31104e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:47.086215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:47.141881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1021/2021] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:47.149537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253481f-a215-4de2-84ac-2555e3756e8e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.149860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253481f-a215-4de2-84ac-2555e3756e8e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.150147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253481f-a215-4de2-84ac-2555e3756e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.169363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1253481f-a215-4de2-84ac-2555e3756e8e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:47.170647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1253481f-a215-4de2-84ac-2555e3756e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:47.178786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1001/2022] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:47.183586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c757e-c731-4d50-8b3f-c700b31104e9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:47.194901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c757e-c731-4d50-8b3f-c700b31104e9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:47.200057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:47.200742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.201032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.201315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.202088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:13:47.202404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:47.204031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.217209 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1022/2023] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:13:47.221908 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253481f-a215-4de2-84ac-2555e3756e8e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:47.239864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253481f-a215-4de2-84ac-2555e3756e8e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:47.244585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:47.246122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.246401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.246672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.247361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:47.249260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.278112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1002/2024] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:47 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:47.282839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:47.299759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9f98f-b420-4c35-b4a8-3b10424331f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:47.358201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:47.358886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.359178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.359462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.360159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:47.362145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.382708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1023/2025] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:47 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:47.387325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:47.398308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79126152-455a-4b41-8bdc-b3daee9958db admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:47.403444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:47.404114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.404400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.404681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.405405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:47.407027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.417692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1003/2026] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:47.422384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:47.439711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d9294-9cd5-462e-9550-5d785b8ebf70 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:47.444462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a87162-c693-46e0-89ce-1ab2346f77f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:47.445175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a87162-c693-46e0-89ce-1ab2346f77f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.445464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a87162-c693-46e0-89ce-1ab2346f77f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.445779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a87162-c693-46e0-89ce-1ab2346f77f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.446512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a87162-c693-46e0-89ce-1ab2346f77f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:47.448663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a87162-c693-46e0-89ce-1ab2346f77f2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.458963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1024/2027] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:47.463293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:47.474012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d155f-cb3c-4e25-9c80-cf74fd86d406 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:47.478604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:47.479527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.479802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.480072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] PATH_INFO: `/v3/projects/aed429e2356c4c7a83872d82c0540b66/users/578e9a80847d4fb18e9ea43a3ed7e8f5/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.495190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed429e2356c4c7a83872d82c0540b66, user_id=578e9a80847d4fb18e9ea43a3ed7e8f5, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:47.497959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.511564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1004/2028] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:47 2022] PUT /identity/v3/projects/aed429e2356c4c7a83872d82c0540b66/users/578e9a80847d4fb18e9ea43a3ed7e8f5/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:13:47.516779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74db8c-5238-460d-82e3-07b42414539d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.517077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74db8c-5238-460d-82e3-07b42414539d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.517421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74db8c-5238-460d-82e3-07b42414539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.535413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bc74db8c-5238-460d-82e3-07b42414539d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:47.536620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bc74db8c-5238-460d-82e3-07b42414539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:47.615370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1025/2029] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:47.625465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:47.627273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:47.685896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73460aa7-abc1-4c93-ae25-c73e0d2b2b71 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:47.689432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:47.690132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:47.690419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:47.690700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:47.701845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:47.703285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:47.713061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1005/2030] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:52.357566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b0373-b65a-4e9f-9e0f-797f6326fdc9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.357566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b0373-b65a-4e9f-9e0f-797f6326fdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.357566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b0373-b65a-4e9f-9e0f-797f6326fdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.397219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da5b0373-b65a-4e9f-9e0f-797f6326fdc9 None None] MFA Rules not processed for user `578e9a80847d4fb18e9ea43a3ed7e8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:52.399309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da5b0373-b65a-4e9f-9e0f-797f6326fdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:52.422003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:52.492919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1026/2031] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:52.497988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.508297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428593fb-8b3d-410d-b004-bcea33aac313 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.513659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.514436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.514788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.515139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.515934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.517933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.559651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1006/2032] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:13:52.563712 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b0373-b65a-4e9f-9e0f-797f6326fdc9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.574994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b0373-b65a-4e9f-9e0f-797f6326fdc9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.637608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.638233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.638520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.638804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.639506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.641806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.666078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1027/2033] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:13:52 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:13:52.668188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.673608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:52.680181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e268f-fb3a-412a-9fde-02aa82ed0251 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.687079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.687757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.688043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.688323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.689134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.690865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.706845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1007/2034] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:52.708373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.725636 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7ee70-5793-4f9b-bf7f-e378dfb7f2cd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.732706 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.733948 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.734575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.734575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] PATH_INFO: `/v3/projects/628ab5a8f33e40048cfc723209f0280e/users/ecd4e8fc68384c6c9c0b4f22f3c9c84d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.753102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=628ab5a8f33e40048cfc723209f0280e, user_id=ecd4e8fc68384c6c9c0b4f22f3c9c84d, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.756184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.779488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1028/2035] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:52 2022] PUT /identity/v3/projects/628ab5a8f33e40048cfc723209f0280e/users/ecd4e8fc68384c6c9c0b4f22f3c9c84d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:52.783610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.794778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8556ad-7673-475e-b80d-29c4da992063 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.870117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef4676-9e23-435c-bf33-f43ba74326cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.870821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4676-9e23-435c-bf33-f43ba74326cf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.871165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4676-9e23-435c-bf33-f43ba74326cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.871447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4676-9e23-435c-bf33-f43ba74326cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.872220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef4676-9e23-435c-bf33-f43ba74326cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.873975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef4676-9e23-435c-bf33-f43ba74326cf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.888786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1008/2036] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:13:52.890074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:52.903610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aad8d-3343-40c8-9230-82612aa54a2d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:52.908512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:52.909422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:52.910094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:52.910390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] PATH_INFO: `/v3/projects/628ab5a8f33e40048cfc723209f0280e/users/ecd4e8fc68384c6c9c0b4f22f3c9c84d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:52.912578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=628ab5a8f33e40048cfc723209f0280e, user_id=ecd4e8fc68384c6c9c0b4f22f3c9c84d, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:52.915320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:52.927339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1029/2037] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:13:52 2022] PUT /identity/v3/projects/628ab5a8f33e40048cfc723209f0280e/users/ecd4e8fc68384c6c9c0b4f22f3c9c84d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:53.649116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dd88f-ef87-4bea-983c-94ba4e4e38da None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:53.650168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dd88f-ef87-4bea-983c-94ba4e4e38da None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:53.651239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3dd88f-ef87-4bea-983c-94ba4e4e38da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:53.686423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bc3dd88f-ef87-4bea-983c-94ba4e4e38da None None] MFA Rules not processed for user `1a98ea9b90174a24b57b3875e9df2ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:53.687683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bc3dd88f-ef87-4bea-983c-94ba4e4e38da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:53.759497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1009/2038] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:13:57.240116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:57.243082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:57.407859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a31a12-b950-43fd-9e14-daf96d5c2670 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:57.417092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:57.425363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:57.425671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:57.425978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] PATH_INFO: `/v3/users/414bb0cab9194ee4a098461d1a1fbb13` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:57.428731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414bb0cab9194ee4a098461d1a1fbb13)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:57.430731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:57.449612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:57.561953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414bb0cab9194ee4a098461d1a1fbb13'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:57.563266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414bb0cab9194ee4a098461d1a1fbb13'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:57.564130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414bb0cab9194ee4a098461d1a1fbb13'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:57.594332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414bb0cab9194ee4a098461d1a1fbb13'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:57.610619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414bb0cab9194ee4a098461d1a1fbb13'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:57.645473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:57.646451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:57.647082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:57.648857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1030/2039] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:57 2022] DELETE /identity/v3/users/414bb0cab9194ee4a098461d1a1fbb13 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:57.677047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:13:57.693699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:57.785330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1010/2040] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:13:57.887419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:57.889048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:58.032186 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e42f27-1f5b-4f83-b280-45b6550ad006 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:58.051426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:58.052102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:58.053007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:58.053459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:58.099386 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:58.101038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:58.133788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1031/2041] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:13:59.663734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:59.665789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:13:59.694447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2e400-411b-4857-b597-d4510c682c44 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:59.711060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:59.723263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:59.723809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:59.724539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] PATH_INFO: `/v3/projects/afb4aa091d4c4f9aa666b72519cc9565` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:59.726762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb4aa091d4c4f9aa666b72519cc9565)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:59.729198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:13:59.800051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:13:59.847277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb4aa091d4c4f9aa666b72519cc9565', 'request_id': 'req-d7919ece-55af-482f-b10a-2e2a3e1b4091'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:59.861314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb4aa091d4c4f9aa666b72519cc9565', 'request_id': 'req-d7919ece-55af-482f-b10a-2e2a3e1b4091'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:59.873763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb4aa091d4c4f9aa666b72519cc9565', 'request_id': 'req-d7919ece-55af-482f-b10a-2e2a3e1b4091'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:59.886474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] The token cache is being invalidate because project afb4aa091d4c4f9aa666b72519cc9565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:13:59.887081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:59.887947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:13:59.890319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1011/2042] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:13:59 2022] DELETE /identity/v3/projects/afb4aa091d4c4f9aa666b72519cc9565 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:13:59.894104 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:13:59.905125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b38348-9409-4b1c-8a77-76def608da1d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:13:59.980871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:13:59.981796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:13:59.982270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:13:59.982694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] PATH_INFO: `/v3/users/c6d722263c684655afa7d020faa015c7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:13:59.984170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d722263c684655afa7d020faa015c7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:13:59.987150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:00.048901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:00.158667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d722263c684655afa7d020faa015c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.164399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d722263c684655afa7d020faa015c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.165457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d722263c684655afa7d020faa015c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.183098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d722263c684655afa7d020faa015c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.206483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4f860d9e-5837-4ad0-8803-a5fa3bc4d62f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d722263c684655afa7d020faa015c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.273045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:00.274963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:00.298833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1032/2043] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:13:59 2022] DELETE /identity/v3/users/c6d722263c684655afa7d020faa015c7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:00.300166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919ece-55af-482f-b10a-2e2a3e1b4091 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:00.503725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:00.506453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:00.506821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:00.508692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] PATH_INFO: `/v3/users/47c5a97e2b2a462e92cadbaf8759cb77` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:00.510575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c5a97e2b2a462e92cadbaf8759cb77)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:00.514636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:00.651403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c5a97e2b2a462e92cadbaf8759cb77'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.655307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c5a97e2b2a462e92cadbaf8759cb77'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.672911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c5a97e2b2a462e92cadbaf8759cb77'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.678614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c5a97e2b2a462e92cadbaf8759cb77'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.706577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b005392-9f25-423b-a8dd-ab72f5dad3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c5a97e2b2a462e92cadbaf8759cb77'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:00.752798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1012/2044] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:00 2022] DELETE /identity/v3/users/47c5a97e2b2a462e92cadbaf8759cb77 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:00.757884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:00.758509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:00.758794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:00.793320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:00.794667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:00.904691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:00.906622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:00.908159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:00.954905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] MFA Rules not processed for user `ecd4e8fc68384c6c9c0b4f22f3c9c84d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:00.956406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:00.967656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1033/2045] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:00.987385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:00.988913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:00.993111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:01.117000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1013/2046] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:01.138682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:01.140720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:01.147662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f1ce2-8531-4b39-b1b0-c7513568f031 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:01.153398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:01.154261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.154549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.154827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.176255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:01.178068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:01.206446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1034/2047] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:14:01.320536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:01.322136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:01.341455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca94cef-17c0-4d2e-bbc1-fac020aae07d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:01.356374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde0c44-5236-465d-8926-4823e5a9e50c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:01.356560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e566b-e693-44a0-8cdd-60ff2b24936a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:01.357058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde0c44-5236-465d-8926-4823e5a9e50c service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.357351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde0c44-5236-465d-8926-4823e5a9e50c service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.357637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde0c44-5236-465d-8926-4823e5a9e50c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.359802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6fde0c44-5236-465d-8926-4823e5a9e50c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:01.365862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:01.366974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.367367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.367923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] PATH_INFO: `/v3/projects/d3a03a4d3617423eb0dad1f31db24ce7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.369682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a03a4d3617423eb0dad1f31db24ce7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:01.372508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:01.438582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:01.444813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a03a4d3617423eb0dad1f31db24ce7', 'request_id': 'req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:01.482505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a03a4d3617423eb0dad1f31db24ce7', 'request_id': 'req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:01.493450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde0c44-5236-465d-8926-4823e5a9e50c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:01.495458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde0c44-5236-465d-8926-4823e5a9e50c service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:01.496803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a03a4d3617423eb0dad1f31db24ce7', 'request_id': 'req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:01.514826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1014/2048] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:01.538164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] The token cache is being invalidate because project d3a03a4d3617423eb0dad1f31db24ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:01.540673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:01.543093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8deadefa-2ea4-4f7c-a6c1-fe4c7433a419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:01.546217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1035/2049] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:01 2022] DELETE /identity/v3/projects/d3a03a4d3617423eb0dad1f31db24ce7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:01.551476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cab0e-13cb-4082-96d8-d02457a85213 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.551910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cab0e-13cb-4082-96d8-d02457a85213 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.552323 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cab0e-13cb-4082-96d8-d02457a85213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.584945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8b3cab0e-13cb-4082-96d8-d02457a85213 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:01.587011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8b3cab0e-13cb-4082-96d8-d02457a85213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:01.673008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:01.696865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1015/2050] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:01.699250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596661-0b03-43c4-b689-e3984b7565a9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.700103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596661-0b03-43c4-b689-e3984b7565a9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.700559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f596661-0b03-43c4-b689-e3984b7565a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.731189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f596661-0b03-43c4-b689-e3984b7565a9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:01.732423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f596661-0b03-43c4-b689-e3984b7565a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:01.746140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1036/2051] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:01.749153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cab0e-13cb-4082-96d8-d02457a85213 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:01.762796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cab0e-13cb-4082-96d8-d02457a85213 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:01.770117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:01.771001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.772220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.772811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.773876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:01.774985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:01.777454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:01.803982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1016/2052] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:01.810023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596661-0b03-43c4-b689-e3984b7565a9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:01.824610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f596661-0b03-43c4-b689-e3984b7565a9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:01.831385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:01.832135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.832426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.832709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.833838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:01.838886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:01.878463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1037/2053] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:01 2022] POST /identity/v3/projects => generated 363 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:01.881248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:01.898872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99842dc1-7cb4-4627-ba85-33a16b84b8d1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:01.988704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:01.989189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:01.989685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:01.990187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:01.991261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:01.995260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.035708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1017/2054] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:01 2022] POST /identity/v3/users => generated 282 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:02.038112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.059033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f193d6-4576-40df-8c4e-2b79de9756e8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.068772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.070297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.071253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:02.071817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.072945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.073348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.074303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.076279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.092277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b0236-e721-45bd-a911-3a286e67e75a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.102774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.104465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.104754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.105035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] PATH_INFO: `/v3/projects/9e222bd5af6e4f1c823147c870262708` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.114983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1038/2055] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:02.118792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.120547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e222bd5af6e4f1c823147c870262708)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.123155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.140110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ecc861-a46a-4349-a0e5-06f24c27da7b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.151972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.152664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.152954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.153235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.154032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.155656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.176503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1039/2056] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:02.182890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.208348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135dfd7-cccf-4a46-b599-13230d9d110b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.241904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e222bd5af6e4f1c823147c870262708', 'request_id': 'req-f022b9fd-d3b6-4716-923c-fbdc71de7413'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:02.258463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e222bd5af6e4f1c823147c870262708', 'request_id': 'req-f022b9fd-d3b6-4716-923c-fbdc71de7413'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:02.271255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e222bd5af6e4f1c823147c870262708', 'request_id': 'req-f022b9fd-d3b6-4716-923c-fbdc71de7413'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:02.297649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] The token cache is being invalidate because project 9e222bd5af6e4f1c823147c870262708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:02.298463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:02.299479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f022b9fd-d3b6-4716-923c-fbdc71de7413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:02.302072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1018/2057] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:02 2022] DELETE /identity/v3/projects/9e222bd5af6e4f1c823147c870262708 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:02.307373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01439a-9af3-4b8e-8f0a-f9211b3d640f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.307987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01439a-9af3-4b8e-8f0a-f9211b3d640f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.308520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01439a-9af3-4b8e-8f0a-f9211b3d640f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.321348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbaf28-0f44-40c1-b1e3-6e791d252aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.323128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbaf28-0f44-40c1-b1e3-6e791d252aee admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.323338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbaf28-0f44-40c1-b1e3-6e791d252aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.323462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbaf28-0f44-40c1-b1e3-6e791d252aee admin admin] PATH_INFO: `/v3/projects/32b8db6af2064495ab15e1089bef6b29/users/715735043a9c4ed0a8afe969396cc351/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.346880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9f01439a-9af3-4b8e-8f0a-f9211b3d640f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:02.348716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f01439a-9af3-4b8e-8f0a-f9211b3d640f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:02.349659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbaf28-0f44-40c1-b1e3-6e791d252aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=32b8db6af2064495ab15e1089bef6b29, user_id=715735043a9c4ed0a8afe969396cc351, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.352726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbaf28-0f44-40c1-b1e3-6e791d252aee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.361154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1019/2058] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:02.369161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14a8b6-1526-49b1-ae46-25a770e301f8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.369815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14a8b6-1526-49b1-ae46-25a770e301f8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.370396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14a8b6-1526-49b1-ae46-25a770e301f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.382643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1040/2059] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:02 2022] PUT /identity/v3/projects/32b8db6af2064495ab15e1089bef6b29/users/715735043a9c4ed0a8afe969396cc351/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:02.394564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0a14a8b6-1526-49b1-ae46-25a770e301f8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:02.396734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0a14a8b6-1526-49b1-ae46-25a770e301f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:02.398782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.399218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.399634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.517139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] MFA Rules not processed for user `715735043a9c4ed0a8afe969396cc351`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:02.518041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:02.518502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:02.545016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1020/2060] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:02.550518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14a8b6-1526-49b1-ae46-25a770e301f8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.565081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14a8b6-1526-49b1-ae46-25a770e301f8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.575902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.576623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.577278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.577278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.577944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:02.578258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.586013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.611830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1021/2061] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:02.613252 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.639373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ad2fb-256c-4e4b-99d2-5d936e84d3aa admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.652862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.654048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.654716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.655327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.656719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.659189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.689150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:02.699507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1022/2062] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:02 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:02.714618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.714879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1041/2063] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:02.719147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.721078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:02.757312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593377d1-458b-4e7a-a8d2-6d5af5f5eadb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.825016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858c12b-f17d-4f42-82ab-cf670160b07a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.831292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.833133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.833524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.834254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.834998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.838205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.871086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1023/2064] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:02 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:02.883386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.884016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.887087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.887377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.889879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.891031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.895188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.905510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dbece-c066-40ee-bf49-a33ea4400312 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:02.920346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:02.924004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:02.924856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:02.925387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:02.927560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:02.929235 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:02.949711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1024/2065] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:02.960842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.963591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1042/2066] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:02 2022] POST /identity/v3/projects => generated 363 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:02.967557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:02.991129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb1bfe-629b-431c-bf56-39f5bdb7a8f9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.049378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27e44f-12f2-46b6-a4b1-926c07c92c70 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.055732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.056640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.057261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.057688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.058689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.060881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.098673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1043/2067] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:02 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:03.103358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:03.114506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.116196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.116761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.116943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c92b82-47ee-4a73-8850-07bb110dfd3d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.117348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.118594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.122019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.125042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.126147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.127275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.127275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.128280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.131674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.146178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1044/2068] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:03.150722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:03.153784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1025/2069] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:03.158961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:03.163286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64df34-655f-46d7-9a88-34d56e32d533 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.176404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a89f0-c45f-418c-962d-6d6782ffe93e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.185098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.185837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8fefb-a8d3-4150-9a48-731f1c4f982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.187936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.190207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.190520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] PATH_INFO: `/v3/projects/f5f12839c03947c78316152046eec95b/users/509b08c8623c4f6ab0cfd972f65e1859/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.195395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8fefb-a8d3-4150-9a48-731f1c4f982e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.195566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8fefb-a8d3-4150-9a48-731f1c4f982e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.195857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8fefb-a8d3-4150-9a48-731f1c4f982e admin admin] PATH_INFO: `/v3/projects/c0b109792d384af7b975f3f5f0189f33/users/7b3bc37d220947d395686939f4488247/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.227474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f12839c03947c78316152046eec95b, user_id=509b08c8623c4f6ab0cfd972f65e1859, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.229236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8fefb-a8d3-4150-9a48-731f1c4f982e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b109792d384af7b975f3f5f0189f33, user_id=7b3bc37d220947d395686939f4488247, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.233541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.234380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8fefb-a8d3-4150-9a48-731f1c4f982e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.265169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1026/2070] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:03 2022] PUT /identity/v3/projects/c0b109792d384af7b975f3f5f0189f33/users/7b3bc37d220947d395686939f4488247/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:03.269149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.269441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.269825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.271781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1045/2071] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:03 2022] PUT /identity/v3/projects/f5f12839c03947c78316152046eec95b/users/509b08c8623c4f6ab0cfd972f65e1859/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:03.276043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:03.294476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:03.328926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:03.337073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] MFA Rules not processed for user `7b3bc37d220947d395686939f4488247`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:03.338898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:03.370505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600362-a40e-4e27-834e-d956ce4157e7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.377295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.378024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.378318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.378603 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.379496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.381334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.397906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1046/2072] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:03.403679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:03.421692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8106e-1b9a-4388-9ff5-968a1cfbd90b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.429002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720f3e5-8935-4419-81f8-7831311a695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.430358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720f3e5-8935-4419-81f8-7831311a695e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.430902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720f3e5-8935-4419-81f8-7831311a695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.431407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720f3e5-8935-4419-81f8-7831311a695e admin admin] PATH_INFO: `/v3/projects/f5f12839c03947c78316152046eec95b/users/509b08c8623c4f6ab0cfd972f65e1859/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.436057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720f3e5-8935-4419-81f8-7831311a695e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f12839c03947c78316152046eec95b, user_id=509b08c8623c4f6ab0cfd972f65e1859, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.440071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720f3e5-8935-4419-81f8-7831311a695e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.463863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1047/2073] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:03 2022] PUT /identity/v3/projects/f5f12839c03947c78316152046eec95b/users/509b08c8623c4f6ab0cfd972f65e1859/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:03.467769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c3cfe-e6ff-4736-b12c-9bdb59c94e19 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.468326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c3cfe-e6ff-4736-b12c-9bdb59c94e19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.468854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c3cfe-e6ff-4736-b12c-9bdb59c94e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.514801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1027/2074] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:03.542008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:03.548638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ae8c3cfe-e6ff-4736-b12c-9bdb59c94e19 None None] MFA Rules not processed for user `509b08c8623c4f6ab0cfd972f65e1859`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:03.550815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:03.553983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ae8c3cfe-e6ff-4736-b12c-9bdb59c94e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:03.702508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af334cff-1179-4b07-9828-dfcd66656909 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.704635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1048/2075] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:03 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:03.706068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.706545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.707319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.753001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] MFA Rules not processed for user `578e9a80847d4fb18e9ea43a3ed7e8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:03.754442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:03.796032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.796880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.797239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.797644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.855836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1049/2076] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:03.856955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:03.860030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:03.882439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9a949-3ab7-4c40-a597-827eab01ca59 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:03.889406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649aa9a-84e4-4154-9801-98f6ab9d5d66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:03.890803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649aa9a-84e4-4154-9801-98f6ab9d5d66 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:03.891168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649aa9a-84e4-4154-9801-98f6ab9d5d66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:03.892118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649aa9a-84e4-4154-9801-98f6ab9d5d66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:03.894288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.896046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.903887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649aa9a-84e4-4154-9801-98f6ab9d5d66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:03.906161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649aa9a-84e4-4154-9801-98f6ab9d5d66 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:03.914847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1028/2077] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:03.923257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1050/2078] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:03 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:05.081494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:05.091154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:05.179334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c5c55-03e8-4c9a-be67-fb10b88497a1 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:05.186441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:05.187594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:05.187976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:05.188390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] PATH_INFO: `/v3/users/715735043a9c4ed0a8afe969396cc351` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:05.189816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] RBAC: Authorizing `identity:delete_user(user_id=715735043a9c4ed0a8afe969396cc351)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:05.192170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:05.269882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715735043a9c4ed0a8afe969396cc351'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:05.270204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715735043a9c4ed0a8afe969396cc351'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:05.292581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715735043a9c4ed0a8afe969396cc351'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:05.292581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715735043a9c4ed0a8afe969396cc351'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:05.330718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715735043a9c4ed0a8afe969396cc351'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:05.350262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1029/2079] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:05 2022] DELETE /identity/v3/users/715735043a9c4ed0a8afe969396cc351 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:05.355066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:05.355573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:05.355664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:05.388318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:05.394002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:05.451924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:05.505821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1051/2080] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:05.517143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:05.519185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:05.627331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467988cb-2ea0-426b-b49d-2736aaf1affa admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:05.632536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:05.633212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:05.633498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:05.633857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:05.644392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:05.646104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:05.661595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1030/2081] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:06.104635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:06.108764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:06.120369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99954c-b26a-483d-ac1e-eca432963154 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:06.125539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:06.126479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:06.126764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:06.127043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] PATH_INFO: `/v3/projects/32b8db6af2064495ab15e1089bef6b29` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:06.128440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32b8db6af2064495ab15e1089bef6b29)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:06.136291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:06.203139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32b8db6af2064495ab15e1089bef6b29', 'request_id': 'req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.216262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32b8db6af2064495ab15e1089bef6b29', 'request_id': 'req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.223879 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32b8db6af2064495ab15e1089bef6b29', 'request_id': 'req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.236263 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] The token cache is being invalidate because project 32b8db6af2064495ab15e1089bef6b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:06.236562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.237238 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.239375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1052/2082] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:06 2022] DELETE /identity/v3/projects/32b8db6af2064495ab15e1089bef6b29 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:06.244680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:06.254786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15bb8f-4337-484e-8508-18dbd3627582 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:06.286033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:06.319082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:06.319961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:06.320246 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:06.320525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] PATH_INFO: `/v3/users/509b08c8623c4f6ab0cfd972f65e1859` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:06.321800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] RBAC: Authorizing `identity:delete_user(user_id=509b08c8623c4f6ab0cfd972f65e1859)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:06.323743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:06.372786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509b08c8623c4f6ab0cfd972f65e1859'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.373469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509b08c8623c4f6ab0cfd972f65e1859'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.380800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509b08c8623c4f6ab0cfd972f65e1859'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.381280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509b08c8623c4f6ab0cfd972f65e1859'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.388317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-eaddd6b1-9a79-4cd0-a0c2-d4b1168b4643 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509b08c8623c4f6ab0cfd972f65e1859'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.406601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1031/2083] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:06 2022] DELETE /identity/v3/users/509b08c8623c4f6ab0cfd972f65e1859 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:06.749497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:06.750607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:06.769291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7598a9-727e-4aa3-ab02-9eacadc60fd6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:06.844350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:06.845213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:06.845495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:06.845946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] PATH_INFO: `/v3/projects/f5f12839c03947c78316152046eec95b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:06.847377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f12839c03947c78316152046eec95b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:06.849399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:06.909421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f12839c03947c78316152046eec95b', 'request_id': 'req-01058900-0d81-4c21-96c0-c7d59f333aa6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.918105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f12839c03947c78316152046eec95b', 'request_id': 'req-01058900-0d81-4c21-96c0-c7d59f333aa6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.925047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f12839c03947c78316152046eec95b', 'request_id': 'req-01058900-0d81-4c21-96c0-c7d59f333aa6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.926335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:06.939989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] The token cache is being invalidate because project f5f12839c03947c78316152046eec95b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:06.940353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.941112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-01058900-0d81-4c21-96c0-c7d59f333aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:06.942998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1053/2084] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:06 2022] DELETE /identity/v3/projects/f5f12839c03947c78316152046eec95b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:06.948215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163cf9-745b-4ff5-9a8c-fb6df3e20d79 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:06.948507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163cf9-745b-4ff5-9a8c-fb6df3e20d79 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:06.948785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44163cf9-745b-4ff5-9a8c-fb6df3e20d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:06.967236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-44163cf9-745b-4ff5-9a8c-fb6df3e20d79 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:06.968549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-44163cf9-745b-4ff5-9a8c-fb6df3e20d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:07.034368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1032/2085] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:07.043543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbeb35a-6821-4b3c-bcd3-b78e00706b16 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.043842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbeb35a-6821-4b3c-bcd3-b78e00706b16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.044157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbeb35a-6821-4b3c-bcd3-b78e00706b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.063419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ccbeb35a-6821-4b3c-bcd3-b78e00706b16 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:07.064631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ccbeb35a-6821-4b3c-bcd3-b78e00706b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:07.073337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1054/2086] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:07.077459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44163cf9-745b-4ff5-9a8c-fb6df3e20d79 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.088887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44163cf9-745b-4ff5-9a8c-fb6df3e20d79 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.093671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.094377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.094665 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.094947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.095684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:07.095993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.097902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.110157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1033/2087] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:07.114608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbeb35a-6821-4b3c-bcd3-b78e00706b16 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.124263 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbeb35a-6821-4b3c-bcd3-b78e00706b16 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.128866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.129511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.129827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.130182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.130883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.132789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.159307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1055/2088] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:07 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:07.163400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.173405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aba4f1-0b0b-4f4a-859c-cdb6fddac0b9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.208639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:07.242782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.243463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.243757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.244039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.245251 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.247216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.278816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1034/2089] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:07 2022] POST /identity/v3/users => generated 282 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:07.280073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.291920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579eae3-faa1-4722-8897-ecdc72428eab admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.296833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.297558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.297883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.298160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.298891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.300504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.311393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1056/2090] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:07.315403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.326027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65651efd-bf40-43db-a9ce-6412a06982f6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.330635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044e7ca-cadf-4a73-893d-2def0b6b3856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.331375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044e7ca-cadf-4a73-893d-2def0b6b3856 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.331735 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044e7ca-cadf-4a73-893d-2def0b6b3856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.332105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044e7ca-cadf-4a73-893d-2def0b6b3856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.332899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044e7ca-cadf-4a73-893d-2def0b6b3856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.334606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044e7ca-cadf-4a73-893d-2def0b6b3856 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.345823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1035/2091] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:07.350913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.360786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48062945-505e-45a8-ba79-3f8f7a4b8a60 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.365375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.366459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.366842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.367074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] PATH_INFO: `/v3/projects/d5babd895d4a441290f1f5f14f942599/users/2cbd6f2f8d554cb49f865bbc1f9ff23c/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.384449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5babd895d4a441290f1f5f14f942599, user_id=2cbd6f2f8d554cb49f865bbc1f9ff23c, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.387710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.402998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1057/2092] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:07 2022] PUT /identity/v3/projects/d5babd895d4a441290f1f5f14f942599/users/2cbd6f2f8d554cb49f865bbc1f9ff23c/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:07.413458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472fae2-937e-4150-9bee-d5113b083ec4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.413774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472fae2-937e-4150-9bee-d5113b083ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.414063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472fae2-937e-4150-9bee-d5113b083ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.441204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6472fae2-937e-4150-9bee-d5113b083ec4 None None] MFA Rules not processed for user `2cbd6f2f8d554cb49f865bbc1f9ff23c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:07.442437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6472fae2-937e-4150-9bee-d5113b083ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:07.510917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1036/2093] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:07.515425 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.577515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c988c6-97bd-4fc5-b4ff-fe02d70002ae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.582349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.583061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.583355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.583638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.584333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.586331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.613619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1058/2094] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:07 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:07.617671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472fae2-937e-4150-9bee-d5113b083ec4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.628459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472fae2-937e-4150-9bee-d5113b083ec4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.685708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.686407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.686696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.686980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.687683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.689617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.710060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1037/2095] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:07 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:07.714629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.725655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d62be1-cf47-4bf7-a3d7-4d6fedc0ab50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.730366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.731034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.731326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.731612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.732373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.734024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.745138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1059/2096] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:07.749404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.760470 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674a6ef-88d1-4cf4-ab25-fd792d8ed615 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.765366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.766317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.766607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.766888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] PATH_INFO: `/v3/projects/eb27a56388e046b480274fbd009e2a9f/users/366adea40c7648cebf3ec461a58d0230/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.783941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb27a56388e046b480274fbd009e2a9f, user_id=366adea40c7648cebf3ec461a58d0230, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.786716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.803145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1038/2097] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:07 2022] PUT /identity/v3/projects/eb27a56388e046b480274fbd009e2a9f/users/366adea40c7648cebf3ec461a58d0230/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:07.808024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.818359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722b114-1551-417a-bece-2df549fe68f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.882661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2bb7e3-5f61-468e-8061-668c1fff174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.885483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bb7e3-5f61-468e-8061-668c1fff174c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.885819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bb7e3-5f61-468e-8061-668c1fff174c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.886106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2bb7e3-5f61-468e-8061-668c1fff174c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.886852 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bb7e3-5f61-468e-8061-668c1fff174c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.888759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2bb7e3-5f61-468e-8061-668c1fff174c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.890447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:07.904434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1060/2098] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:07.908991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:07.919405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1205754-68dd-4133-9ba0-e9640eab97aa admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:07.924627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:07.925508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.925813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.926086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] PATH_INFO: `/v3/projects/eb27a56388e046b480274fbd009e2a9f/users/366adea40c7648cebf3ec461a58d0230/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:07.937239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb27a56388e046b480274fbd009e2a9f, user_id=366adea40c7648cebf3ec461a58d0230, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:07.937239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:07.939861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:07.968896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1039/2099] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:07 2022] PUT /identity/v3/projects/eb27a56388e046b480274fbd009e2a9f/users/366adea40c7648cebf3ec461a58d0230/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:07.983672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a2e8-88ba-4d13-a21f-d4e1cf4272c2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:07.983969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a2e8-88ba-4d13-a21f-d4e1cf4272c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:07.984251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a2e8-88ba-4d13-a21f-d4e1cf4272c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:08.023544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6650a2e8-88ba-4d13-a21f-d4e1cf4272c2 None None] MFA Rules not processed for user `366adea40c7648cebf3ec461a58d0230`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:08.024927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6650a2e8-88ba-4d13-a21f-d4e1cf4272c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:08.102316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1061/2100] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:08.114909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:08.116964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:08.189627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf2d19-337c-4e1f-b0f2-6e0897b47dfe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:08.193475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:08.194040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:08.194413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:08.194672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:08.205842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:08.207298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:08.218443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1040/2101] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:08.293544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a2e8-88ba-4d13-a21f-d4e1cf4272c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:08.303639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a2e8-88ba-4d13-a21f-d4e1cf4272c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:08.307651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817ff05-c394-4928-af7e-643cf6ef5539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:08.308329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ff05-c394-4928-af7e-643cf6ef5539 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:08.308610 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ff05-c394-4928-af7e-643cf6ef5539 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:08.308896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ff05-c394-4928-af7e-643cf6ef5539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:08.376189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817ff05-c394-4928-af7e-643cf6ef5539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:08.377645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817ff05-c394-4928-af7e-643cf6ef5539 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:08.387966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1062/2102] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:18.213517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:18.383416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=64 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:23.391975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:23.393709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:23.475397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f124e-63fc-4182-b4a4-ad2049dc4fda service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:23.481888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:23.482696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:23.482985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:23.483408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] PATH_INFO: `/v3/users/7b3bc37d220947d395686939f4488247` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:23.484641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3bc37d220947d395686939f4488247)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:23.486681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:23.549972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3bc37d220947d395686939f4488247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:23.549972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3bc37d220947d395686939f4488247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:23.553887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3bc37d220947d395686939f4488247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:23.554631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3bc37d220947d395686939f4488247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:23.562641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3bc37d220947d395686939f4488247'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:23.580998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1041/2103] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:23 2022] DELETE /identity/v3/users/7b3bc37d220947d395686939f4488247 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:23.585474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:23.586030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:23.586124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:23.609935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:23.611195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:23.678616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1063/2104] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:23.689152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:23.690592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:23.750862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5863289-cf97-4d51-a3f0-4b9849ff5fc0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:23.754721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92da0e-414d-4b33-aa70-a90a0f14ef5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:23.755381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92da0e-414d-4b33-aa70-a90a0f14ef5f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:23.755671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92da0e-414d-4b33-aa70-a90a0f14ef5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:23.755949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92da0e-414d-4b33-aa70-a90a0f14ef5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:23.767411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92da0e-414d-4b33-aa70-a90a0f14ef5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:23.768873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92da0e-414d-4b33-aa70-a90a0f14ef5f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:23.779180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1042/2105] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:24.058788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.060215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:24.077336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55046a68-8d4e-4892-ab42-be09d5deb6b8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.082186 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.083273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.083368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.083676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] PATH_INFO: `/v3/projects/c0b109792d384af7b975f3f5f0189f33` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.084862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b109792d384af7b975f3f5f0189f33)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.086803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.135245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b109792d384af7b975f3f5f0189f33', 'request_id': 'req-6588c63e-70c6-4461-a629-de4f22dcf4ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:24.144532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b109792d384af7b975f3f5f0189f33', 'request_id': 'req-6588c63e-70c6-4461-a629-de4f22dcf4ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:24.151585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b109792d384af7b975f3f5f0189f33', 'request_id': 'req-6588c63e-70c6-4461-a629-de4f22dcf4ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:24.162830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] The token cache is being invalidate because project c0b109792d384af7b975f3f5f0189f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:24.163163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:24.163832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6588c63e-70c6-4461-a629-de4f22dcf4ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:24.165434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1064/2106] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:24 2022] DELETE /identity/v3/projects/c0b109792d384af7b975f3f5f0189f33 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:24.170481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f2db7-6e9a-4d8d-87e8-8387de2c654d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.170774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f2db7-6e9a-4d8d-87e8-8387de2c654d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.171054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f2db7-6e9a-4d8d-87e8-8387de2c654d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.189121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-785f2db7-6e9a-4d8d-87e8-8387de2c654d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:24.190559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-785f2db7-6e9a-4d8d-87e8-8387de2c654d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:24.211950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:24.269856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1043/2107] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:24.278214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018d7b-ce0e-41dd-b308-3f613c354b83 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.278214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018d7b-ce0e-41dd-b308-3f613c354b83 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.278214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018d7b-ce0e-41dd-b308-3f613c354b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.302051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-57018d7b-ce0e-41dd-b308-3f613c354b83 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:24.303198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-57018d7b-ce0e-41dd-b308-3f613c354b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:24.311784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1065/2108] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:24.318772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f2db7-6e9a-4d8d-87e8-8387de2c654d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.328887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f2db7-6e9a-4d8d-87e8-8387de2c654d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.334090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.334614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.334912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.335192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.335930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:24.336238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.337931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.351783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1044/2109] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:24.356224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57018d7b-ce0e-41dd-b308-3f613c354b83 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.366797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57018d7b-ce0e-41dd-b308-3f613c354b83 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.372831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.373514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.373843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.374122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.374882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.376828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.405524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1066/2110] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:24 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:24.409701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.420359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dfacc-8953-403c-9040-a9c020eb7893 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.479987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.480680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.480964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.481240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.481964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.483878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.504530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1045/2111] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:24 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:24.509035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.519875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633fcf5-38dd-4ef9-9039-70928bd2dcd2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.525346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.526081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.526364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.526640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.527364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.529017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.547643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1067/2112] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:24.552047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.567438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f518a48-25f3-4ff0-9f51-4fb884148402 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.572867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b64036-90f7-42fa-8deb-e3c6114b6e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.573532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64036-90f7-42fa-8deb-e3c6114b6e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.573856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64036-90f7-42fa-8deb-e3c6114b6e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.574136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b64036-90f7-42fa-8deb-e3c6114b6e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.574863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b64036-90f7-42fa-8deb-e3c6114b6e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.576516 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b64036-90f7-42fa-8deb-e3c6114b6e92 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.587017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1046/2113] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:24.591259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.601868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea38f-a0a0-4849-a1fd-637e8ddbcf9e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.607837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.608740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.609032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.609309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] PATH_INFO: `/v3/projects/12aa35389847441e803933fe1b887979/users/fe5c504939a6434a9855548a32575c66/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.628570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12aa35389847441e803933fe1b887979, user_id=fe5c504939a6434a9855548a32575c66, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.631308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.648462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1068/2114] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:24 2022] PUT /identity/v3/projects/12aa35389847441e803933fe1b887979/users/fe5c504939a6434a9855548a32575c66/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:24.658982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae485f8-2f73-4800-8d23-e3315f843068 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.658982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae485f8-2f73-4800-8d23-e3315f843068 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.659382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae485f8-2f73-4800-8d23-e3315f843068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.688712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2ae485f8-2f73-4800-8d23-e3315f843068 None None] MFA Rules not processed for user `fe5c504939a6434a9855548a32575c66`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:24.690083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2ae485f8-2f73-4800-8d23-e3315f843068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:24.766807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1047/2115] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:24.771594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.795341 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:24.851798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b49174-e9f8-487a-9665-37648dc9c5a3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.856989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.857653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.857993 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.858277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.858989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.860897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:24.887690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1069/2116] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:24 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:24.891993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae485f8-2f73-4800-8d23-e3315f843068 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:24.906020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae485f8-2f73-4800-8d23-e3315f843068 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:24.923465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:24.977103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:24.977821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:24.978171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:24.978972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:24.979672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:24.981675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.004746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1048/2117] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:24 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:25.008756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.019072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d07b44-1ccf-4c87-82ea-f5690cd43558 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.024200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.025402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.025691 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.026000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.026727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.028344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.038963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1070/2118] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:25.043724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.054041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae178047-2ae0-40b6-a984-e62d3ac500f9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.059652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e47859-9cb8-431f-a26d-161fe502ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.060269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e47859-9cb8-431f-a26d-161fe502ef4f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.060552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e47859-9cb8-431f-a26d-161fe502ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.060834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e47859-9cb8-431f-a26d-161fe502ef4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.061552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e47859-9cb8-431f-a26d-161fe502ef4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.063199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e47859-9cb8-431f-a26d-161fe502ef4f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.073423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1049/2119] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:25.077608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.088424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1182cb7-d903-4518-9a45-e76250ff7823 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.092989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.093932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.094221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.094499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] PATH_INFO: `/v3/projects/bd193fc1a77040a7b73db8b10f4ddee9/users/6c8c60d4d88b47dc93b7b654cbf7ee68/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.110034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd193fc1a77040a7b73db8b10f4ddee9, user_id=6c8c60d4d88b47dc93b7b654cbf7ee68, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.112692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.128241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1071/2120] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:25 2022] PUT /identity/v3/projects/bd193fc1a77040a7b73db8b10f4ddee9/users/6c8c60d4d88b47dc93b7b654cbf7ee68/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:25.155461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a93c62-392e-415d-b51e-be50ec315957 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.155461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a93c62-392e-415d-b51e-be50ec315957 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.155461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a93c62-392e-415d-b51e-be50ec315957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.175932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b1a93c62-392e-415d-b51e-be50ec315957 None None] MFA Rules not processed for user `6c8c60d4d88b47dc93b7b654cbf7ee68`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:25.177141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b1a93c62-392e-415d-b51e-be50ec315957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:25.243948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1050/2121] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:25.248649 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.313597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9a359-25ea-43b5-adef-fe90e207940d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.318343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.319065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.319371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.319685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.320421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.323068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.349852 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1072/2122] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:25 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:25.354028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a93c62-392e-415d-b51e-be50ec315957 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.364976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a93c62-392e-415d-b51e-be50ec315957 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.426734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.427574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.428500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.428500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.428868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.431157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.452812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1051/2123] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:25 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:25.456764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.468437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db319793-aa28-4f65-95bf-fa6c867a4287 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.474927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.475604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.475991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.476196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.477106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.478959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.490726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1073/2124] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:25.494879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.505145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9dd9f-2f52-43bf-994c-e5dfcd46d823 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.510914 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.512378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.513026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.513533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] PATH_INFO: `/v3/projects/193db868e13547698190c85306ddfcc1/users/e46e1fc954d646da9710042f22b4f668/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.537152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:25.540953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=193db868e13547698190c85306ddfcc1, user_id=e46e1fc954d646da9710042f22b4f668, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.544517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.559657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1052/2125] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:25 2022] PUT /identity/v3/projects/193db868e13547698190c85306ddfcc1/users/e46e1fc954d646da9710042f22b4f668/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:25.564686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.576823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ad88-854d-437c-b461-c322b1232280 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.627125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:25.644625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf7bba5-34d7-4a42-b75f-e09064480de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.645613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7bba5-34d7-4a42-b75f-e09064480de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.645613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7bba5-34d7-4a42-b75f-e09064480de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.645910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7bba5-34d7-4a42-b75f-e09064480de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.646624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7bba5-34d7-4a42-b75f-e09064480de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.648294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf7bba5-34d7-4a42-b75f-e09064480de5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.659290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1074/2126] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:25.663297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.675502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f21bf4-fdff-429c-8842-6cb7dcdb17ca admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.680531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.681683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.681987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.682507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] PATH_INFO: `/v3/projects/193db868e13547698190c85306ddfcc1/users/e46e1fc954d646da9710042f22b4f668/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.684621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] RBAC: Authorizing `identity:create_grant(project_id=193db868e13547698190c85306ddfcc1, user_id=e46e1fc954d646da9710042f22b4f668, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.702573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.725758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1053/2127] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:25 2022] PUT /identity/v3/projects/193db868e13547698190c85306ddfcc1/users/e46e1fc954d646da9710042f22b4f668/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:25.733034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515799-df31-4556-9039-27ab04c2699a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.733330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515799-df31-4556-9039-27ab04c2699a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.733615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66515799-df31-4556-9039-27ab04c2699a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.762482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-66515799-df31-4556-9039-27ab04c2699a None None] MFA Rules not processed for user `e46e1fc954d646da9710042f22b4f668`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:25.763732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66515799-df31-4556-9039-27ab04c2699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:25.845415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1075/2128] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:25.853991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:25.856175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:25.919376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917a67-a32e-4813-bbb7-47b1ce95115e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:25.923152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d4b82-c0c8-40e0-ba97-c8be46f7bf67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:25.923801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d4b82-c0c8-40e0-ba97-c8be46f7bf67 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:25.924087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d4b82-c0c8-40e0-ba97-c8be46f7bf67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:25.924367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d4b82-c0c8-40e0-ba97-c8be46f7bf67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:25.982033 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d4b82-c0c8-40e0-ba97-c8be46f7bf67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:25.983481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d4b82-c0c8-40e0-ba97-c8be46f7bf67 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:25.993690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1054/2129] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:26.187108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515799-df31-4556-9039-27ab04c2699a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:26.188535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66515799-df31-4556-9039-27ab04c2699a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:26.264074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66515799-df31-4556-9039-27ab04c2699a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:26.268924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:26.270040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:26.270374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:26.270483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] PATH_INFO: `/v3/users/2cbd6f2f8d554cb49f865bbc1f9ff23c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:26.271716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cbd6f2f8d554cb49f865bbc1f9ff23c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:26.273824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:26.331656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cbd6f2f8d554cb49f865bbc1f9ff23c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:26.332294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cbd6f2f8d554cb49f865bbc1f9ff23c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:26.333472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cbd6f2f8d554cb49f865bbc1f9ff23c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:26.342504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cbd6f2f8d554cb49f865bbc1f9ff23c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:26.350792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cbd6f2f8d554cb49f865bbc1f9ff23c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:26.355596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:26.371530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1076/2130] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:26 2022] DELETE /identity/v3/users/2cbd6f2f8d554cb49f865bbc1f9ff23c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:26.376351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:26.376621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:26.376907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:26.395169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:26.396383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:26.458288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1055/2131] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:26.468518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:26.470087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:26.528361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9067f-5e4b-48e7-a248-92bcf89af1d2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:26.532049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:26.532693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:26.532977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:26.533251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:26.542659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:26.544091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:26.553433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1077/2132] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:26.989785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:26.991780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:27.006357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80285a-eb45-47f3-977e-246eaf88d9df None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:27.013370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:27.023966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:27.024260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:27.024979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] PATH_INFO: `/v3/projects/d5babd895d4a441290f1f5f14f942599` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:27.026401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5babd895d4a441290f1f5f14f942599)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:27.028690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:27.065762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:27.082830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5babd895d4a441290f1f5f14f942599', 'request_id': 'req-081460ce-1a0f-41eb-a7d2-a3ee033208ac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.090511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5babd895d4a441290f1f5f14f942599', 'request_id': 'req-081460ce-1a0f-41eb-a7d2-a3ee033208ac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.097563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5babd895d4a441290f1f5f14f942599', 'request_id': 'req-081460ce-1a0f-41eb-a7d2-a3ee033208ac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.112433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] The token cache is being invalidate because project d5babd895d4a441290f1f5f14f942599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:27.113867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.114610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.116393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1056/2133] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:26 2022] DELETE /identity/v3/projects/d5babd895d4a441290f1f5f14f942599 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:27.120374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:27.130817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ba8e6-5adf-4722-be34-31ff96c25493 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:27.197697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:27.198603 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:27.198886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:27.199162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] PATH_INFO: `/v3/users/366adea40c7648cebf3ec461a58d0230` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:27.200371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=366adea40c7648cebf3ec461a58d0230)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:27.202321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:27.250517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366adea40c7648cebf3ec461a58d0230'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.251187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366adea40c7648cebf3ec461a58d0230'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.251791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366adea40c7648cebf3ec461a58d0230'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.261077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366adea40c7648cebf3ec461a58d0230'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.271764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366adea40c7648cebf3ec461a58d0230'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.292094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1078/2134] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:27 2022] DELETE /identity/v3/users/366adea40c7648cebf3ec461a58d0230 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:27.595779 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:27.597817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:27.607182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:27.608726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:27.622732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:27.683898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081460ce-1a0f-41eb-a7d2-a3ee033208ac admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:27.706980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f6f6f-3217-49e9-94a1-28f6821dd6ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:27.712103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:27.713306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:27.713689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:27.714063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] PATH_INFO: `/v3/projects/eb27a56388e046b480274fbd009e2a9f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:27.715346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb27a56388e046b480274fbd009e2a9f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:27.717461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:27.742423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aae3ef-17d7-4ba5-8fb9-43323bff2fcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:27.743113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aae3ef-17d7-4ba5-8fb9-43323bff2fcb service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:27.743400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aae3ef-17d7-4ba5-8fb9-43323bff2fcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:27.743746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aae3ef-17d7-4ba5-8fb9-43323bff2fcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:27.745546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-15aae3ef-17d7-4ba5-8fb9-43323bff2fcb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:27.773217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb27a56388e046b480274fbd009e2a9f', 'request_id': 'req-15c9a981-058b-424b-b79a-ebf07df0af46'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.785025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb27a56388e046b480274fbd009e2a9f', 'request_id': 'req-15c9a981-058b-424b-b79a-ebf07df0af46'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.792964 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb27a56388e046b480274fbd009e2a9f', 'request_id': 'req-15c9a981-058b-424b-b79a-ebf07df0af46'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.805773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] The token cache is being invalidate because project eb27a56388e046b480274fbd009e2a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:27.806126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.806814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-15c9a981-058b-424b-b79a-ebf07df0af46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:27.808833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1079/2135] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:27 2022] DELETE /identity/v3/projects/eb27a56388e046b480274fbd009e2a9f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:27.813579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bfe3c-7f0c-4787-8169-bf5b5d276d1b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:27.814701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bfe3c-7f0c-4787-8169-bf5b5d276d1b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:27.814809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bfe3c-7f0c-4787-8169-bf5b5d276d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:27.820640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aae3ef-17d7-4ba5-8fb9-43323bff2fcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:27.823100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aae3ef-17d7-4ba5-8fb9-43323bff2fcb service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:27.833086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1057/2136] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:27.838233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-394bfe3c-7f0c-4787-8169-bf5b5d276d1b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:27.839484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-394bfe3c-7f0c-4787-8169-bf5b5d276d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:27.907714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1080/2137] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:27.917174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ef889-01c7-4f5e-b396-95e1f62fac75 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:27.917346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ef889-01c7-4f5e-b396-95e1f62fac75 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:27.917691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ef889-01c7-4f5e-b396-95e1f62fac75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:27.938212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-865ef889-01c7-4f5e-b396-95e1f62fac75 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:27.939446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-865ef889-01c7-4f5e-b396-95e1f62fac75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:27.951865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1058/2138] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:27.956436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bfe3c-7f0c-4787-8169-bf5b5d276d1b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:27.966809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bfe3c-7f0c-4787-8169-bf5b5d276d1b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:27.975590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:27.976361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:27.976608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:27.976831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:27.977674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:27.978274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:27.980084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.001870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1081/2139] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:28.008411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ef889-01c7-4f5e-b396-95e1f62fac75 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.025099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ef889-01c7-4f5e-b396-95e1f62fac75 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.037138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.037863 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.038152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.038435 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.039147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.041250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.065789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:28.078886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1059/2140] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:27 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:28.079407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.091967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447f437-c9e7-4ea1-b68c-98631c8f742c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.127205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:28.177705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.178599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.178895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.179181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.180052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.182953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.209828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1082/2141] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:28 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:28.211814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.222906 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41d18d-eb2b-4422-8e21-c340781e475d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.228343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.229180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.229612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.229987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.230795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.232407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.245779 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1060/2142] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:28.248500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.260945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93255f5-143f-4b79-9d85-ff7d050da045 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.266986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a00c5-b008-4634-adf6-847a404d3bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.267673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a00c5-b008-4634-adf6-847a404d3bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.267961 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a00c5-b008-4634-adf6-847a404d3bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.268242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a00c5-b008-4634-adf6-847a404d3bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.268971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a00c5-b008-4634-adf6-847a404d3bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.270671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a00c5-b008-4634-adf6-847a404d3bb4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.283838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1083/2143] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:28.288120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.298975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de078b9-fdf8-4418-a46f-f58406f79bed admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.304625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.305599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.306652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.307006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] PATH_INFO: `/v3/projects/2c7e084cb0c4475796ef2160db9291b9/users/e7f0b4ce977e46b28f8aeec456659bea/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.329445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7e084cb0c4475796ef2160db9291b9, user_id=e7f0b4ce977e46b28f8aeec456659bea, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.332493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.348284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1061/2144] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:28 2022] PUT /identity/v3/projects/2c7e084cb0c4475796ef2160db9291b9/users/e7f0b4ce977e46b28f8aeec456659bea/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:28.359942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.360266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.360595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.404349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] MFA Rules not processed for user `e7f0b4ce977e46b28f8aeec456659bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:28.405599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:28.492836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1084/2145] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:14:28.494880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.530846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.535773 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:28.585075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db710163-8f70-4197-a8e1-0fb776b2aab6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.590723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.591463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.591750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.592030 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.592716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.594705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.602515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0fc6d-e86c-493c-86eb-c2c624c5fe58 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.606422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.607079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.607362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.607645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.609475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:28.625056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1062/2146] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:28 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:28.626154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.657652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:28.679052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.680500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.691225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1085/2147] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:28.701662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b9017-ac86-47ad-8164-5c3def30a695 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.706437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.707103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.707386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.708625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.709329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.711310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.737297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1063/2148] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:28 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:14:28.738498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.750201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092b255-08ec-4728-b8a7-c8383a0f33ed service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.777346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.778423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.778864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.779170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.779920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.781684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.794091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1086/2149] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:28.798243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.811350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a34188-cf1f-4595-9c26-f76785b25f85 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.821765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.821765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.821765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.821765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] PATH_INFO: `/v3/projects/2361dfc7918649c49a4a8a6a034c4100/users/7efa3659caa64551b2196069ff023f04/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.837447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2361dfc7918649c49a4a8a6a034c4100, user_id=7efa3659caa64551b2196069ff023f04, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.840157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.853741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1064/2150] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:28 2022] PUT /identity/v3/projects/2361dfc7918649c49a4a8a6a034c4100/users/7efa3659caa64551b2196069ff023f04/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:14:28.857875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.867802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd422e-02ec-41ec-bb36-eaf824a664d2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.924490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17da51-b2fc-4da7-b1ba-9761df8ce20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.925154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17da51-b2fc-4da7-b1ba-9761df8ce20f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.925438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17da51-b2fc-4da7-b1ba-9761df8ce20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.925740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17da51-b2fc-4da7-b1ba-9761df8ce20f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.926467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17da51-b2fc-4da7-b1ba-9761df8ce20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.928062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17da51-b2fc-4da7-b1ba-9761df8ce20f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.938202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1087/2151] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:28.942308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:28.953074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d1d8-6ecb-421f-b6a3-a7ef610a181e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:28.958256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:28.959139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.959425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.959704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] PATH_INFO: `/v3/projects/2361dfc7918649c49a4a8a6a034c4100/users/7efa3659caa64551b2196069ff023f04/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:28.961779 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2361dfc7918649c49a4a8a6a034c4100, user_id=7efa3659caa64551b2196069ff023f04, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:28.965588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:28.973241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:28.978270 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1065/2152] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:28 2022] PUT /identity/v3/projects/2361dfc7918649c49a4a8a6a034c4100/users/7efa3659caa64551b2196069ff023f04/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:28.989734 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0b5de-52c9-4701-bd87-26b9bfe8f9fe None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:28.989904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0b5de-52c9-4701-bd87-26b9bfe8f9fe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:28.990348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0b5de-52c9-4701-bd87-26b9bfe8f9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:29.017861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5ec0b5de-52c9-4701-bd87-26b9bfe8f9fe None None] MFA Rules not processed for user `7efa3659caa64551b2196069ff023f04`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:29.019507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ec0b5de-52c9-4701-bd87-26b9bfe8f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:29.090573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1088/2153] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:29.101553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:29.103411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:29.163465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c9f5b-45ec-432f-8013-10d8d9421a4a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:29.167108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:29.167745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:29.168026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:29.168300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:29.224818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:29.226405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:29.235695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1066/2154] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:39.086854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=76 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:39.231414 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=48 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:43.307365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0b5de-52c9-4701-bd87-26b9bfe8f9fe None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:43.318924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0b5de-52c9-4701-bd87-26b9bfe8f9fe None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:43.322541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e818a94-db7d-457a-b4aa-5ff275b51e56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:43.323223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e818a94-db7d-457a-b4aa-5ff275b51e56 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:43.323512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e818a94-db7d-457a-b4aa-5ff275b51e56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:43.323795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e818a94-db7d-457a-b4aa-5ff275b51e56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:43.333874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e818a94-db7d-457a-b4aa-5ff275b51e56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:43.335312 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e818a94-db7d-457a-b4aa-5ff275b51e56 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:43.345929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1089/2155] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:44.342495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.343970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:44.410537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7a1a6-a596-4632-b98f-029f15ac5651 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:44.415377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:44.416244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.416556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.416957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] PATH_INFO: `/v3/users/fe5c504939a6434a9855548a32575c66` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:44.418418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5c504939a6434a9855548a32575c66)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:44.420402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:44.471605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5c504939a6434a9855548a32575c66'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:44.472518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5c504939a6434a9855548a32575c66'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:44.480455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5c504939a6434a9855548a32575c66'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:44.481150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5c504939a6434a9855548a32575c66'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:44.488616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5c504939a6434a9855548a32575c66'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:44.508370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1067/2156] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:44 2022] DELETE /identity/v3/users/fe5c504939a6434a9855548a32575c66 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:44.513073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.513571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.513666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:44.532037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:44.533255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:44.601040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1090/2157] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:44.611686 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:44.613399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:44.676552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94500fb4-fb4b-42a3-8821-8dfe95266d37 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:44.680413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:44.681144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:44.681432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:44.681711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:44.692684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:44.694486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:44.704591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1068/2158] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:45.030421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:45.031889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:45.053465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d97c73-49a0-4273-a6c3-99af376273f0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:45.067898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:45.069317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:45.069317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:45.069573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] PATH_INFO: `/v3/projects/12aa35389847441e803933fe1b887979` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.070966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12aa35389847441e803933fe1b887979)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:45.074568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:45.161035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12aa35389847441e803933fe1b887979', 'request_id': 'req-56ed0c0c-7d53-4234-991c-7db76afb1b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.170374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12aa35389847441e803933fe1b887979', 'request_id': 'req-56ed0c0c-7d53-4234-991c-7db76afb1b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.177994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12aa35389847441e803933fe1b887979', 'request_id': 'req-56ed0c0c-7d53-4234-991c-7db76afb1b97'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.190197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] The token cache is being invalidate because project 12aa35389847441e803933fe1b887979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:45.190511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.191167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.193071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1091/2159] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:45 2022] DELETE /identity/v3/projects/12aa35389847441e803933fe1b887979 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:45.197604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:45.207321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73254d04-6d02-45b3-b8ea-c1bde050b3fd service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:45.228454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:45.272152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:45.273043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:45.273333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:45.273617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] PATH_INFO: `/v3/users/6c8c60d4d88b47dc93b7b654cbf7ee68` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.274899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8c60d4d88b47dc93b7b654cbf7ee68)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:45.277397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:45.340505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8c60d4d88b47dc93b7b654cbf7ee68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.341182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8c60d4d88b47dc93b7b654cbf7ee68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.348610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8c60d4d88b47dc93b7b654cbf7ee68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.349285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8c60d4d88b47dc93b7b654cbf7ee68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.357905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8c60d4d88b47dc93b7b654cbf7ee68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.376093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1069/2160] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:45 2022] DELETE /identity/v3/users/6c8c60d4d88b47dc93b7b654cbf7ee68 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:45.670602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:45.672445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:45.687774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed0c0c-7d53-4234-991c-7db76afb1b97 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:45.775838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:45.777108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:45.777593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:45.778090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] PATH_INFO: `/v3/projects/bd193fc1a77040a7b73db8b10f4ddee9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.779507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd193fc1a77040a7b73db8b10f4ddee9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:45.781513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:45.794390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:45.840501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd193fc1a77040a7b73db8b10f4ddee9', 'request_id': 'req-31c534f7-e547-406c-9df3-7ff539c29f7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.849309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd193fc1a77040a7b73db8b10f4ddee9', 'request_id': 'req-31c534f7-e547-406c-9df3-7ff539c29f7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.857273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd193fc1a77040a7b73db8b10f4ddee9', 'request_id': 'req-31c534f7-e547-406c-9df3-7ff539c29f7f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.869630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] The token cache is being invalidate because project bd193fc1a77040a7b73db8b10f4ddee9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:45.869954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.870939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:45.872669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1092/2161] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:45 2022] DELETE /identity/v3/projects/bd193fc1a77040a7b73db8b10f4ddee9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:45.877509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:45.882275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:45.911047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab11ac8-8ff5-40f9-8dec-0fef724a31b4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:45.982911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:45.984020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:45.984303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:45.984581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] PATH_INFO: `/v3/users/e46e1fc954d646da9710042f22b4f668` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:45.985848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46e1fc954d646da9710042f22b4f668)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:45.987770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:46.039727 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:46.052729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46e1fc954d646da9710042f22b4f668'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.053409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46e1fc954d646da9710042f22b4f668'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.061387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46e1fc954d646da9710042f22b4f668'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.062060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46e1fc954d646da9710042f22b4f668'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.070243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d6273779-5aa2-4b94-92f0-adde2cc8e5a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46e1fc954d646da9710042f22b4f668'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.089629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1070/2162] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:45 2022] DELETE /identity/v3/users/e46e1fc954d646da9710042f22b4f668 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:46.457997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:46.462565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:46.549468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c534f7-e547-406c-9df3-7ff539c29f7f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:46.608044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:46.608918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:46.609245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:46.609607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] PATH_INFO: `/v3/projects/193db868e13547698190c85306ddfcc1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:46.610966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=193db868e13547698190c85306ddfcc1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:46.612890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:46.662881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193db868e13547698190c85306ddfcc1', 'request_id': 'req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.664564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:46.671190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193db868e13547698190c85306ddfcc1', 'request_id': 'req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.679672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193db868e13547698190c85306ddfcc1', 'request_id': 'req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.693278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] The token cache is being invalidate because project 193db868e13547698190c85306ddfcc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:46.693576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.694277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f3ae3fa-d79a-4d98-8130-fb5fab4edb0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:46.696055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1093/2163] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:46 2022] DELETE /identity/v3/projects/193db868e13547698190c85306ddfcc1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:46.702972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0b2d5-4552-4b59-8f5e-fcec83f24b0b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:46.703478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0b2d5-4552-4b59-8f5e-fcec83f24b0b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:46.703556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0b2d5-4552-4b59-8f5e-fcec83f24b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:46.753988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0be0b2d5-4552-4b59-8f5e-fcec83f24b0b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:46.755441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0be0b2d5-4552-4b59-8f5e-fcec83f24b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:46.832865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1071/2164] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:46.842162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd55ed8-759c-4de2-a09d-ff17aaccfb3d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:46.842460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd55ed8-759c-4de2-a09d-ff17aaccfb3d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:46.842751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd55ed8-759c-4de2-a09d-ff17aaccfb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:46.867150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0bd55ed8-759c-4de2-a09d-ff17aaccfb3d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:46.869005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0bd55ed8-759c-4de2-a09d-ff17aaccfb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:46.877308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1094/2165] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:46.881614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0b2d5-4552-4b59-8f5e-fcec83f24b0b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:46.903222 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0b2d5-4552-4b59-8f5e-fcec83f24b0b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:46.919578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:46.920350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:46.920638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:46.920915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:46.921654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:46.922014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:46.923892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:46.940561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1072/2166] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:46.943476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd55ed8-759c-4de2-a09d-ff17aaccfb3d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:46.954164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd55ed8-759c-4de2-a09d-ff17aaccfb3d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:46.961641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:46.963116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:46.963405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:46.963703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:46.964577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:46.966551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:47.010906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1095/2167] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:46 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:47.017496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:47.032293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b4376-3616-4e1a-90ad-bd033ee1076a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:47.103762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:47.104649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:47.105087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:47.105365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:47.106088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:47.115565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:47.145556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1073/2168] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:47 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:47.145907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:47.159119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f34d21-4412-412a-a887-3154670a6365 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:47.163865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:47.164544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:47.164874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:47.165152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:47.165917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:47.167590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:47.181758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1096/2169] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:47.182813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:47.193064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029bcb-94c9-463c-aabc-474d7d8e41fc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:47.198513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f881a00-19a4-4f2c-8534-e294fab1d5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:47.199309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f881a00-19a4-4f2c-8534-e294fab1d5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:47.199596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f881a00-19a4-4f2c-8534-e294fab1d5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:47.199873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f881a00-19a4-4f2c-8534-e294fab1d5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:47.200601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f881a00-19a4-4f2c-8534-e294fab1d5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:47.202265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f881a00-19a4-4f2c-8534-e294fab1d5aa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:47.203948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:47.219644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1074/2170] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:14:47.223767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:47.234359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce34be5-c78f-44f2-99ab-38d0ad0373a1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:47.239973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:47.240849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:47.241236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:47.241416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] PATH_INFO: `/v3/projects/999705a6bba24e4198240e1f8b08c4d9/users/6e0dcd599835413a9271a038bec143a1/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:47.260103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=999705a6bba24e4198240e1f8b08c4d9, user_id=6e0dcd599835413a9271a038bec143a1, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:47.262935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:47.278611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1097/2171] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:47 2022] PUT /identity/v3/projects/999705a6bba24e4198240e1f8b08c4d9/users/6e0dcd599835413a9271a038bec143a1/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:47.290238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:47.290534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:47.290816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:47.320426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] MFA Rules not processed for user `6e0dcd599835413a9271a038bec143a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:47.322259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:47.411264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1075/2172] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:47.429314 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:47.432143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:47.541250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094854a-add3-497b-9374-be11c0d8a970 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:47.544923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:47.545575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:47.545912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:47.546197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:47.555454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:47.556902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:47.568252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1098/2173] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:14:48.669120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:48.671639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:48.760513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468763c-aeb5-4904-8baf-33ec743aa0e2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:48.765026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:48.765927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:48.766215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:48.766499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] PATH_INFO: `/v3/users/5e2021bba1874f0fb4e6904aac576431` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:48.767748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2021bba1874f0fb4e6904aac576431)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:48.769994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:48.827272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2021bba1874f0fb4e6904aac576431'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.827931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2021bba1874f0fb4e6904aac576431'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.835421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2021bba1874f0fb4e6904aac576431'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.836005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2021bba1874f0fb4e6904aac576431'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.844432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b36fb9d1-4a13-40be-be02-55bf94faf000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2021bba1874f0fb4e6904aac576431'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:48.864814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1076/2174] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:14:48 2022] DELETE /identity/v3/users/5e2021bba1874f0fb4e6904aac576431 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:49.058021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.070182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacc0a8-3b73-4aad-9658-8c979a2658f2 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.129298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.130293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.130576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.130853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] PATH_INFO: `/v3/projects/24fb126babee4bbea59923603503a21a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.136171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24fb126babee4bbea59923603503a21a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.138409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.141883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:49.191719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24fb126babee4bbea59923603503a21a', 'request_id': 'req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:49.199083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24fb126babee4bbea59923603503a21a', 'request_id': 'req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:49.205696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24fb126babee4bbea59923603503a21a', 'request_id': 'req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:49.216514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] The token cache is being invalidate because project 24fb126babee4bbea59923603503a21a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:14:49.216807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:49.217445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f67be316-4fa1-4a55-8bfb-9c91ccbd81b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:14:49.219002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1099/2175] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:14:49 2022] DELETE /identity/v3/projects/24fb126babee4bbea59923603503a21a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:49.224377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735f485-afb2-44cd-9ca1-6b3040746e49 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.224675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735f485-afb2-44cd-9ca1-6b3040746e49 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.224961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735f485-afb2-44cd-9ca1-6b3040746e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.232792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:49.244544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c735f485-afb2-44cd-9ca1-6b3040746e49 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:49.246211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c735f485-afb2-44cd-9ca1-6b3040746e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:49.305693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1077/2176] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:49.313441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bcb91-50af-4004-b766-56445e4ea8d3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.313753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bcb91-50af-4004-b766-56445e4ea8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.314052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bcb91-50af-4004-b766-56445e4ea8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.331999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b88bcb91-50af-4004-b766-56445e4ea8d3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:49.333211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b88bcb91-50af-4004-b766-56445e4ea8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:49.342262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1100/2177] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:49.346457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735f485-afb2-44cd-9ca1-6b3040746e49 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.356617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735f485-afb2-44cd-9ca1-6b3040746e49 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.361395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.362085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.362382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.362663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.363423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:14:49.363897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.365415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.377360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1078/2178] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:14:49.381515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bcb91-50af-4004-b766-56445e4ea8d3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.394170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bcb91-50af-4004-b766-56445e4ea8d3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.401645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.402585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.403003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.403478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.404252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.406384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.432470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1101/2179] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:14:49 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:49.436723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.447497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92d564-35b7-4943-960a-50692522152f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.503387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.504132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.504420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.504692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.505371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.507318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.527357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1079/2180] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:14:49 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:14:49.531546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.542937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883078b8-c585-42a0-ac42-94412ac432dc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.547607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.548268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.548555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.548835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.549562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.551224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.562067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1102/2181] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:49.566118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.576684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa200fc-c484-48e0-acf6-95fdafe61b47 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.581242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114e4e64-58e4-445e-8686-568cb482cac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.582037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e4e64-58e4-445e-8686-568cb482cac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.582377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e4e64-58e4-445e-8686-568cb482cac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.582617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e4e64-58e4-445e-8686-568cb482cac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.583333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e4e64-58e4-445e-8686-568cb482cac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.584954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114e4e64-58e4-445e-8686-568cb482cac3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.595022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1080/2182] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:14:49.599407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.609740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c967434-d1cd-45a8-b145-cb3badfb6d16 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.614928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.615797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.616082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.616355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] PATH_INFO: `/v3/projects/ccbee657f36d4102a56bf9e89e243e73/users/5c9ae15e84934f13b5543c1d6706c1bf/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.631323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccbee657f36d4102a56bf9e89e243e73, user_id=5c9ae15e84934f13b5543c1d6706c1bf, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.634004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.647578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1103/2183] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:14:49 2022] PUT /identity/v3/projects/ccbee657f36d4102a56bf9e89e243e73/users/5c9ae15e84934f13b5543c1d6706c1bf/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:14:49.658208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916f18-cf9d-4849-92d4-34c54167aef6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.658499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916f18-cf9d-4849-92d4-34c54167aef6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.658781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916f18-cf9d-4849-92d4-34c54167aef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.685748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cc916f18-cf9d-4849-92d4-34c54167aef6 None None] MFA Rules not processed for user `5c9ae15e84934f13b5543c1d6706c1bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:49.686949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cc916f18-cf9d-4849-92d4-34c54167aef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:49.747531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:14:49.751346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1081/2184] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:49.763110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:14:49.764830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:49.838469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359eebb-fa2c-48a9-8cb7-c1bfcd4a0f76 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:14:49.844095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:14:49.844736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:49.845017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:49.845342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:49.853560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:14:49.855174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:14:49.863964 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1104/2185] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:14:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:14:54.241213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a27dc-d3a4-486f-aa87-d3e6d96cd125 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:14:54.241744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a27dc-d3a4-486f-aa87-d3e6d96cd125 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:14:54.241840 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a27dc-d3a4-486f-aa87-d3e6d96cd125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:14:54.262120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-965a27dc-d3a4-486f-aa87-d3e6d96cd125 None None] MFA Rules not processed for user `6e0dcd599835413a9271a038bec143a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:14:54.263416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-965a27dc-d3a4-486f-aa87-d3e6d96cd125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:14:54.319989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1082/2186] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:14:59.859841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:04.311344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=25 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:18.468282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.561904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dca8a-674b-43b0-9e7c-2fe367354c92 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.567423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.568471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.568735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.569074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] PATH_INFO: `/v3/users/5c9ae15e84934f13b5543c1d6706c1bf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.570516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9ae15e84934f13b5543c1d6706c1bf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.572760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.628982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9ae15e84934f13b5543c1d6706c1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.629934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9ae15e84934f13b5543c1d6706c1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.630658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9ae15e84934f13b5543c1d6706c1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.639417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9ae15e84934f13b5543c1d6706c1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.647666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9ae15e84934f13b5543c1d6706c1bf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:18.675775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1105/2187] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:18 2022] DELETE /identity/v3/users/5c9ae15e84934f13b5543c1d6706c1bf => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:18.680145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.680630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.680713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.698530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:18.699739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:18.768185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1083/2188] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:18.776219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:18.777570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:18.891223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f60e8c-cfb0-4567-b576-1927e8e8473d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:18.895417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679137a-b7fe-45e5-bc26-c2b70987f2e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:18.896080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679137a-b7fe-45e5-bc26-c2b70987f2e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:18.896363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679137a-b7fe-45e5-bc26-c2b70987f2e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:18.896639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679137a-b7fe-45e5-bc26-c2b70987f2e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:18.907716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679137a-b7fe-45e5-bc26-c2b70987f2e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:18.909284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679137a-b7fe-45e5-bc26-c2b70987f2e1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:18.929034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1106/2189] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:19.335881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.342457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:19.362249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400239d-a465-4025-9908-bf3f849fd19c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:19.370076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:19.371084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.371483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.371820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] PATH_INFO: `/v3/projects/ccbee657f36d4102a56bf9e89e243e73` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.373864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccbee657f36d4102a56bf9e89e243e73)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:19.376533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:19.487564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccbee657f36d4102a56bf9e89e243e73', 'request_id': 'req-8ca5b717-1e4c-434e-8889-992a7635027e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:19.496508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccbee657f36d4102a56bf9e89e243e73', 'request_id': 'req-8ca5b717-1e4c-434e-8889-992a7635027e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:19.503934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccbee657f36d4102a56bf9e89e243e73', 'request_id': 'req-8ca5b717-1e4c-434e-8889-992a7635027e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:19.515292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] The token cache is being invalidate because project ccbee657f36d4102a56bf9e89e243e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:19.515666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:19.516409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ca5b717-1e4c-434e-8889-992a7635027e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:19.518076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1084/2190] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:19 2022] DELETE /identity/v3/projects/ccbee657f36d4102a56bf9e89e243e73 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:19.523037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdf545-0213-4b49-b619-eec13d3195ac None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.523621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdf545-0213-4b49-b619-eec13d3195ac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.523621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcdf545-0213-4b49-b619-eec13d3195ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.527160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:19.564477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9fcdf545-0213-4b49-b619-eec13d3195ac None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:19.565998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9fcdf545-0213-4b49-b619-eec13d3195ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:19.641004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1107/2191] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:19.649245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695e50e-ad9d-46e6-acb4-113f74b1be43 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.649539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695e50e-ad9d-46e6-acb4-113f74b1be43 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.649870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695e50e-ad9d-46e6-acb4-113f74b1be43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.668706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0695e50e-ad9d-46e6-acb4-113f74b1be43 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:19.669962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0695e50e-ad9d-46e6-acb4-113f74b1be43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:19.679676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1085/2192] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:19.684564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdf545-0213-4b49-b619-eec13d3195ac None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.700849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcdf545-0213-4b49-b619-eec13d3195ac None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:19.705835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:19.706552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.706846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.707122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.707861 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:19.708174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:19.709830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:19.728535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1108/2193] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:19.729409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695e50e-ad9d-46e6-acb4-113f74b1be43 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.739031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695e50e-ad9d-46e6-acb4-113f74b1be43 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:19.744549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:19.745216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.745536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.745852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.746554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:19.748609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:19.777783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1086/2194] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:19 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:19.781972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.792474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5042d09-0e1b-4e3e-aba3-0603e96e897d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:19.857062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:19.857783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.858444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.858444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.859049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:19.860987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:19.890085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1109/2195] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:19 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:19.893772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.909131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccbf1e3-dca5-4973-ab4b-0eedf14bf91f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:19.917029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:19.917748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.918047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.918326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.919058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:19.920716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:19.937282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1087/2196] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:19.941612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.957855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb389-d13e-4ce2-848e-d66d452f6bb7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:19.963380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34129bea-4fdf-421b-9fd7-8449188b4c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:19.964484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129bea-4fdf-421b-9fd7-8449188b4c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:19.964484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129bea-4fdf-421b-9fd7-8449188b4c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:19.964757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129bea-4fdf-421b-9fd7-8449188b4c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:19.965484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34129bea-4fdf-421b-9fd7-8449188b4c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:19.967155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34129bea-4fdf-421b-9fd7-8449188b4c69 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:19.982208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1110/2197] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:19.984071 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:19.999708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6236e57-6828-432f-b2e6-3d61c623efd6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:20.006134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:20.007294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:20.007626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:20.008462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] PATH_INFO: `/v3/projects/d9e4d20155164b3d9cbe4f0a893e18c7/users/ede7cbb4890e486c8c28a0cb327db115/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:20.024260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e4d20155164b3d9cbe4f0a893e18c7, user_id=ede7cbb4890e486c8c28a0cb327db115, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:20.026957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:20.041325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1088/2198] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:19 2022] PUT /identity/v3/projects/d9e4d20155164b3d9cbe4f0a893e18c7/users/ede7cbb4890e486c8c28a0cb327db115/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:20.052206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f2d5e-f6e4-46f9-8ae9-677e08fe3233 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:20.052505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f2d5e-f6e4-46f9-8ae9-677e08fe3233 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:20.052793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f2d5e-f6e4-46f9-8ae9-677e08fe3233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:20.083716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-907f2d5e-f6e4-46f9-8ae9-677e08fe3233 None None] MFA Rules not processed for user `ede7cbb4890e486c8c28a0cb327db115`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:20.084955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-907f2d5e-f6e4-46f9-8ae9-677e08fe3233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:20.160323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:20.183599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1111/2199] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:20.533487 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:20.535811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:20.562371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:20.658326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38e108-fd3b-4a21-b2ab-7a1571158419 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:20.664275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64219b-9704-4c97-97c2-ab5f57333d03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:20.664922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64219b-9704-4c97-97c2-ab5f57333d03 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:20.665212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64219b-9704-4c97-97c2-ab5f57333d03 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:20.665495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64219b-9704-4c97-97c2-ab5f57333d03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:20.674155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64219b-9704-4c97-97c2-ab5f57333d03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:20.676005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64219b-9704-4c97-97c2-ab5f57333d03 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:20.686247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1089/2200] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:20.913141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f2d5e-f6e4-46f9-8ae9-677e08fe3233 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:20.979524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f2d5e-f6e4-46f9-8ae9-677e08fe3233 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:20.984937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:20.985857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:20.986147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:20.986429 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] PATH_INFO: `/v3/users/ede7cbb4890e486c8c28a0cb327db115` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:20.987750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede7cbb4890e486c8c28a0cb327db115)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:20.989699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:21.037963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede7cbb4890e486c8c28a0cb327db115'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.038632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede7cbb4890e486c8c28a0cb327db115'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.039787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede7cbb4890e486c8c28a0cb327db115'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.046568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede7cbb4890e486c8c28a0cb327db115'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.053339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede7cbb4890e486c8c28a0cb327db115'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.071411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1112/2201] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:20 2022] DELETE /identity/v3/users/ede7cbb4890e486c8c28a0cb327db115 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:21.075994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.076469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.076600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.093891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:21.095095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:21.160611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1090/2202] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:15:21.170940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:21.172348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:21.231591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd24cba-7ee5-4b5c-bfb0-f253fa76adf4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:21.235241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f08d3-ad36-4688-90bd-0baef9a2c476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:21.235901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f08d3-ad36-4688-90bd-0baef9a2c476 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.236320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f08d3-ad36-4688-90bd-0baef9a2c476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.236606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f08d3-ad36-4688-90bd-0baef9a2c476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.247231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f08d3-ad36-4688-90bd-0baef9a2c476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:21.248675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f08d3-ad36-4688-90bd-0baef9a2c476 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:21.260082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1113/2203] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:21.532489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:21.534056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:21.553309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95875982-3380-4923-882d-1fc3f2c31105 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:21.562125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:21.563004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.563486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.563769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] PATH_INFO: `/v3/projects/d9e4d20155164b3d9cbe4f0a893e18c7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.565125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e4d20155164b3d9cbe4f0a893e18c7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:21.567460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:21.618889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e4d20155164b3d9cbe4f0a893e18c7', 'request_id': 'req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.627181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e4d20155164b3d9cbe4f0a893e18c7', 'request_id': 'req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.634427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e4d20155164b3d9cbe4f0a893e18c7', 'request_id': 'req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.646101 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] The token cache is being invalidate because project d9e4d20155164b3d9cbe4f0a893e18c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:21.646393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.647045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ba34ec33-c3f4-469a-adcf-940ffd6b1e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:21.648697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1091/2204] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:21 2022] DELETE /identity/v3/projects/d9e4d20155164b3d9cbe4f0a893e18c7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:21.653769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd36a3-f4fa-4f74-bab6-1d8b3ca136c7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.654072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd36a3-f4fa-4f74-bab6-1d8b3ca136c7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.654350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd36a3-f4fa-4f74-bab6-1d8b3ca136c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.672577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d8bd36a3-f4fa-4f74-bab6-1d8b3ca136c7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:21.673844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8bd36a3-f4fa-4f74-bab6-1d8b3ca136c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:21.680170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:21.739846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1114/2205] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:21.748459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c7fc8-4219-4c8e-8db6-65ed0bb93a98 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.748683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c7fc8-4219-4c8e-8db6-65ed0bb93a98 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.748966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c7fc8-4219-4c8e-8db6-65ed0bb93a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.755286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:21.770780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ad6c7fc8-4219-4c8e-8db6-65ed0bb93a98 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:21.771989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ad6c7fc8-4219-4c8e-8db6-65ed0bb93a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:21.779782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1092/2206] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:15:21.784057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd36a3-f4fa-4f74-bab6-1d8b3ca136c7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:21.794412 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd36a3-f4fa-4f74-bab6-1d8b3ca136c7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:21.799164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:21.799998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.800350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.800570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.801789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:21.802160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:21.803655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:21.816517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1115/2207] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:21.820920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c7fc8-4219-4c8e-8db6-65ed0bb93a98 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:21.830251 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c7fc8-4219-4c8e-8db6-65ed0bb93a98 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:21.835182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:21.835873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.836153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.836425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.837131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:21.839369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:21.865119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1093/2208] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:21 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:21.870094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:21.880178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35ec47-eb0c-46d1-932f-35cf30fbc5af admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:21.935064 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:21.935739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.936011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.936289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.936971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:21.938935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:21.959419 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1116/2209] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:21 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:21.964709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:21.975886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97291a2f-324c-4fcc-97a3-4498dfe8593b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:21.981184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:21.981876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:21.982161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:21.982433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:21.983177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:21.984782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:21.995807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1094/2210] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:21.999922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.010913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97505fc7-91eb-4931-9b87-195294841254 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.015935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d98e15-7f23-4d25-b775-c5d9fddd1993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.016614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d98e15-7f23-4d25-b775-c5d9fddd1993 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.016901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d98e15-7f23-4d25-b775-c5d9fddd1993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.017181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d98e15-7f23-4d25-b775-c5d9fddd1993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.017953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d98e15-7f23-4d25-b775-c5d9fddd1993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.019706 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d98e15-7f23-4d25-b775-c5d9fddd1993 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.032045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1117/2211] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:22.036019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.046954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3203b0-4b67-4954-b2a9-ccb11ef47986 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.051869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.052825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.053642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.054016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] PATH_INFO: `/v3/projects/07d40663ef3d417c977f0ccf3e14c8b0/users/9687f938b71e49f9a162010c7f36874d/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.078567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d40663ef3d417c977f0ccf3e14c8b0, user_id=9687f938b71e49f9a162010c7f36874d, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.081310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.096226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1095/2212] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:22 2022] PUT /identity/v3/projects/07d40663ef3d417c977f0ccf3e14c8b0/users/9687f938b71e49f9a162010c7f36874d/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:22.107545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbd49d-19f0-46f1-a332-66ff81d3ee5d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.107843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbd49d-19f0-46f1-a332-66ff81d3ee5d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.108131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbd49d-19f0-46f1-a332-66ff81d3ee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.135185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6fbd49d-19f0-46f1-a332-66ff81d3ee5d None None] MFA Rules not processed for user `9687f938b71e49f9a162010c7f36874d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:22.136415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6fbd49d-19f0-46f1-a332-66ff81d3ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:22.203160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1118/2213] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:22.207734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.269171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46fd1-d0a7-4a3e-813e-aa728ae099f9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.274586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.275261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.275549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.275896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.276599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.278561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.304163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1096/2214] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:22 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:22.308336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbd49d-19f0-46f1-a332-66ff81d3ee5d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.319929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbd49d-19f0-46f1-a332-66ff81d3ee5d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.324161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:22.385355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.386245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.386542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.386826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.387612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.389530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.411844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1119/2215] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:22 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:22.415775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.426153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc02dfa-e826-4a66-b492-349460c53d9a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.432052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.432817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.433188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.433471 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.434357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.436019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.449134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1097/2216] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:22.453398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.467762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f8e3-c3fb-4f65-9e92-53398a258c82 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.473187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.474361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.474483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.474755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] PATH_INFO: `/v3/projects/e742178e725f45b6971f6caddfa3cb8b/users/bb67f3fe5f5e4c5583855b9a9ca0e21d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.500150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e742178e725f45b6971f6caddfa3cb8b, user_id=bb67f3fe5f5e4c5583855b9a9ca0e21d, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.503073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.517244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1120/2217] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:22 2022] PUT /identity/v3/projects/e742178e725f45b6971f6caddfa3cb8b/users/bb67f3fe5f5e4c5583855b9a9ca0e21d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:22.521096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.532035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123ef0e-9b1c-4553-b661-91ff6d954ab5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.593430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63041d0e-d2c8-4ee4-9f62-0d9b69663565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.594133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63041d0e-d2c8-4ee4-9f62-0d9b69663565 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.594422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63041d0e-d2c8-4ee4-9f62-0d9b69663565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.594702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63041d0e-d2c8-4ee4-9f62-0d9b69663565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.595425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63041d0e-d2c8-4ee4-9f62-0d9b69663565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.597036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63041d0e-d2c8-4ee4-9f62-0d9b69663565 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.608177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1098/2218] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:22.612423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.623573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e035c-06d9-4275-aa0b-e93b6831004f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.628119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.629021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.629308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.629590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] PATH_INFO: `/v3/projects/e742178e725f45b6971f6caddfa3cb8b/users/bb67f3fe5f5e4c5583855b9a9ca0e21d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.631590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e742178e725f45b6971f6caddfa3cb8b, user_id=bb67f3fe5f5e4c5583855b9a9ca0e21d, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.634278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.645259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1121/2219] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:22 2022] PUT /identity/v3/projects/e742178e725f45b6971f6caddfa3cb8b/users/bb67f3fe5f5e4c5583855b9a9ca0e21d/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:22.656742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49a10-7e28-40f4-83d6-09d71d49cb1d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.657040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49a10-7e28-40f4-83d6-09d71d49cb1d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.657347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49a10-7e28-40f4-83d6-09d71d49cb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.675939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:22.687138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cdd49a10-7e28-40f4-83d6-09d71d49cb1d None None] MFA Rules not processed for user `bb67f3fe5f5e4c5583855b9a9ca0e21d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:22.688456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cdd49a10-7e28-40f4-83d6-09d71d49cb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:22.763945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1099/2220] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:22.776256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:22.777954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:22.870377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151d636-3564-48cf-889f-a78195dad467 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:22.874147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa55d35-9dda-4f0c-99e5-9b633286dc86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:22.874804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55d35-9dda-4f0c-99e5-9b633286dc86 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:22.875119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55d35-9dda-4f0c-99e5-9b633286dc86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:22.875364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa55d35-9dda-4f0c-99e5-9b633286dc86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:22.885220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa55d35-9dda-4f0c-99e5-9b633286dc86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:22.886693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa55d35-9dda-4f0c-99e5-9b633286dc86 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:22.896476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1122/2221] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:23.299463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd49a10-7e28-40f4-83d6-09d71d49cb1d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:23.382206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd49a10-7e28-40f4-83d6-09d71d49cb1d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:23.391043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:23.392577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:23.393125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:23.393638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] PATH_INFO: `/v3/users/9687f938b71e49f9a162010c7f36874d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:23.395526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9687f938b71e49f9a162010c7f36874d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:23.398069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:23.449987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9687f938b71e49f9a162010c7f36874d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:23.450806 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9687f938b71e49f9a162010c7f36874d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:23.457704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9687f938b71e49f9a162010c7f36874d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:23.458389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9687f938b71e49f9a162010c7f36874d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:23.467621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9687f938b71e49f9a162010c7f36874d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:23.486571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1100/2222] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:23 2022] DELETE /identity/v3/users/9687f938b71e49f9a162010c7f36874d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:23.491019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:23.491474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:23.491776 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:23.512300 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:23.513579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:23.555147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:23.584752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1123/2223] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:23.594106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:23.595514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:23.611004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:23.668292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438560f-97ad-435d-89e6-437490bc44d5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:23.672132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:23.672781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:23.673074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:23.673356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:23.684010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:23.685844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:23.695456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1101/2224] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:24.024664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:24.026155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:24.038227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34239d-222b-46e4-98f4-a46d78de8c28 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:24.043212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:24.044132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:24.044420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:24.044703 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] PATH_INFO: `/v3/projects/07d40663ef3d417c977f0ccf3e14c8b0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:24.045978 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d40663ef3d417c977f0ccf3e14c8b0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:24.048058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:24.104141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d40663ef3d417c977f0ccf3e14c8b0', 'request_id': 'req-797a8838-e595-48b9-88a0-ea939c568918'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.111841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d40663ef3d417c977f0ccf3e14c8b0', 'request_id': 'req-797a8838-e595-48b9-88a0-ea939c568918'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.118319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d40663ef3d417c977f0ccf3e14c8b0', 'request_id': 'req-797a8838-e595-48b9-88a0-ea939c568918'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.128922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] The token cache is being invalidate because project 07d40663ef3d417c977f0ccf3e14c8b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:24.129217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.129989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.131675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1124/2225] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:24 2022] DELETE /identity/v3/projects/07d40663ef3d417c977f0ccf3e14c8b0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:24.139630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:24.150638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ef9c7-9576-45d3-96e5-8c325dfb7d2b service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:24.209865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:24.211312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:24.211642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:24.211919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] PATH_INFO: `/v3/users/bb67f3fe5f5e4c5583855b9a9ca0e21d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:24.213115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb67f3fe5f5e4c5583855b9a9ca0e21d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:24.215089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:24.262670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb67f3fe5f5e4c5583855b9a9ca0e21d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.263310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb67f3fe5f5e4c5583855b9a9ca0e21d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.270026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb67f3fe5f5e4c5583855b9a9ca0e21d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.270642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb67f3fe5f5e4c5583855b9a9ca0e21d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.278233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fe6e9ae8-c0d6-4c2c-a2e1-0153c3b6ef30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb67f3fe5f5e4c5583855b9a9ca0e21d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.296525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1102/2226] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:24 2022] DELETE /identity/v3/users/bb67f3fe5f5e4c5583855b9a9ca0e21d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:24.660392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:24.662037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:24.676500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a8838-e595-48b9-88a0-ea939c568918 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:24.745298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:24.746206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:24.746496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:24.746779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] PATH_INFO: `/v3/projects/e742178e725f45b6971f6caddfa3cb8b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:24.748598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e742178e725f45b6971f6caddfa3cb8b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:24.750574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:24.803440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e742178e725f45b6971f6caddfa3cb8b', 'request_id': 'req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.811586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e742178e725f45b6971f6caddfa3cb8b', 'request_id': 'req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.819686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e742178e725f45b6971f6caddfa3cb8b', 'request_id': 'req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.825174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:24.833571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] The token cache is being invalidate because project e742178e725f45b6971f6caddfa3cb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:24.833897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.834744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3ae5d328-881b-4ad8-bb5d-bd9420f9128a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:24.837121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1125/2227] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:24 2022] DELETE /identity/v3/projects/e742178e725f45b6971f6caddfa3cb8b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:24.841996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851772b5-0c17-43ac-bf82-2545f266df1d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:24.842495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851772b5-0c17-43ac-bf82-2545f266df1d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:24.842628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851772b5-0c17-43ac-bf82-2545f266df1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:24.860838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-851772b5-0c17-43ac-bf82-2545f266df1d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:24.862141 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-851772b5-0c17-43ac-bf82-2545f266df1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:24.892576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:24.924442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1103/2228] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:24.932849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefff808-bf7f-4236-97de-6d6abbd31629 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:24.933142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefff808-bf7f-4236-97de-6d6abbd31629 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:24.933426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefff808-bf7f-4236-97de-6d6abbd31629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:24.952679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-eefff808-bf7f-4236-97de-6d6abbd31629 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:24.953917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eefff808-bf7f-4236-97de-6d6abbd31629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:24.962447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1126/2229] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:24.966633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851772b5-0c17-43ac-bf82-2545f266df1d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:24.977418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851772b5-0c17-43ac-bf82-2545f266df1d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:24.981960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:24.982607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:24.982890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:24.983166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:24.983887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:24.984195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:24.985820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:24.998200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1104/2230] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:25.001835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefff808-bf7f-4236-97de-6d6abbd31629 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.011780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefff808-bf7f-4236-97de-6d6abbd31629 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.016532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.017190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.017475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.017784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.018688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.020424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.047461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1127/2231] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:25 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:25.051952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.062621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d815c92-0d78-448f-b539-f6b4f8daff69 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.123977 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.124662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.124952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.125236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.125966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.127896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.150942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1105/2232] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:25 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:25.155141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.166878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864a9d1-89f5-4cfb-b9ab-36824bc69926 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.171457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.172121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.172405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.172682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.173405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.175258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.186860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1128/2233] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:25.190594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.205127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29e4f4-7bff-4b56-8a44-3fe58ba65b07 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.211591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf13b38-0f54-42a1-954f-e84c345632ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.212273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf13b38-0f54-42a1-954f-e84c345632ee admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.212725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf13b38-0f54-42a1-954f-e84c345632ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.213009 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf13b38-0f54-42a1-954f-e84c345632ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.213757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf13b38-0f54-42a1-954f-e84c345632ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.215922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf13b38-0f54-42a1-954f-e84c345632ee admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.231897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1106/2234] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:25.233527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.243796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e68c7-3076-484e-9615-a2fdbc8f0c44 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.248438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.249343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.249618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.249928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] PATH_INFO: `/v3/projects/48a5ae63688a455191d02cbdb7a416a7/users/2cec1cf986564f5ca8e9b9c4187d4da7/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.265196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a5ae63688a455191d02cbdb7a416a7, user_id=2cec1cf986564f5ca8e9b9c4187d4da7, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.267903 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.281738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1129/2235] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:25 2022] PUT /identity/v3/projects/48a5ae63688a455191d02cbdb7a416a7/users/2cec1cf986564f5ca8e9b9c4187d4da7/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:25.292482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a7c7c-b9ac-4305-8ff7-6e3878497d6f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.292777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a7c7c-b9ac-4305-8ff7-6e3878497d6f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.293060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a7c7c-b9ac-4305-8ff7-6e3878497d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.319520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-267a7c7c-b9ac-4305-8ff7-6e3878497d6f None None] MFA Rules not processed for user `2cec1cf986564f5ca8e9b9c4187d4da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:25.320734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-267a7c7c-b9ac-4305-8ff7-6e3878497d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:25.388677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1107/2236] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:25.393092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.455161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52460d-3ed1-434a-a9ad-4d012ef539b1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.459853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.460517 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.461334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.461617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.462353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.464351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.490403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1130/2237] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:25 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:25.495139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a7c7c-b9ac-4305-8ff7-6e3878497d6f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.505982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a7c7c-b9ac-4305-8ff7-6e3878497d6f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.534031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:25.564873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.565646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.565960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.566238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.566925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.568865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.589635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1108/2238] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:25 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:25.593494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.606327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291443e-9d10-4396-b7c9-8aee942f41f3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.611188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.611860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.612147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.612432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.613168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.614855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.630208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1131/2239] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:25.634462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.645460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a216ed1-d987-41f4-97d7-7fdb8b38a442 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.650304 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.651197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.651486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.651768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] PATH_INFO: `/v3/projects/78347d18859e44a6bfd5e96ae0e339b0/users/c8bf188847fa42bd989ed49d2cc8550e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.668459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78347d18859e44a6bfd5e96ae0e339b0, user_id=c8bf188847fa42bd989ed49d2cc8550e, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.671177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.685633 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1109/2240] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:25 2022] PUT /identity/v3/projects/78347d18859e44a6bfd5e96ae0e339b0/users/c8bf188847fa42bd989ed49d2cc8550e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:25.689665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.699831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6fd261-4bba-4275-8770-85e0c4b5fbc1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.755232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f02b8b-c4a5-4e12-97c1-db9558e5766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.755889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f02b8b-c4a5-4e12-97c1-db9558e5766e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.756171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f02b8b-c4a5-4e12-97c1-db9558e5766e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.756446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f02b8b-c4a5-4e12-97c1-db9558e5766e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.757166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f02b8b-c4a5-4e12-97c1-db9558e5766e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.759394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f02b8b-c4a5-4e12-97c1-db9558e5766e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.769184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1132/2241] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:25.773409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.783884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec982b-851d-467a-9183-469d3e640ed1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:25.789022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:25.789950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.790231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.790546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] PATH_INFO: `/v3/projects/78347d18859e44a6bfd5e96ae0e339b0/users/c8bf188847fa42bd989ed49d2cc8550e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.792565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=78347d18859e44a6bfd5e96ae0e339b0, user_id=c8bf188847fa42bd989ed49d2cc8550e, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:25.796171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:25.807531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1110/2242] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:25 2022] PUT /identity/v3/projects/78347d18859e44a6bfd5e96ae0e339b0/users/c8bf188847fa42bd989ed49d2cc8550e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:25.819407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7393c-698d-4a24-aa34-b18a44e3cafc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:25.820727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7393c-698d-4a24-aa34-b18a44e3cafc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:25.821012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7393c-698d-4a24-aa34-b18a44e3cafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:25.826858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:25.850356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-56e7393c-698d-4a24-aa34-b18a44e3cafc None None] MFA Rules not processed for user `c8bf188847fa42bd989ed49d2cc8550e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:25.851584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56e7393c-698d-4a24-aa34-b18a44e3cafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:25.929144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1133/2243] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:25.940381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:25.942448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:26.022600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9bd00-3cf9-4f6d-ace0-4328a465f32a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:26.026940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7d47cf-2203-442e-ab54-0be0a81b8f3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:26.028484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d47cf-2203-442e-ab54-0be0a81b8f3f service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:26.028692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d47cf-2203-442e-ab54-0be0a81b8f3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:26.029063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d47cf-2203-442e-ab54-0be0a81b8f3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:26.041648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d47cf-2203-442e-ab54-0be0a81b8f3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:26.043220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7d47cf-2203-442e-ab54-0be0a81b8f3f service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:26.054390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1111/2244] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:26.192574 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7393c-698d-4a24-aa34-b18a44e3cafc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:26.271002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7393c-698d-4a24-aa34-b18a44e3cafc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:26.277872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:26.278780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:26.279063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:26.279655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] PATH_INFO: `/v3/users/2cec1cf986564f5ca8e9b9c4187d4da7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:26.281354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cec1cf986564f5ca8e9b9c4187d4da7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:26.283761 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:26.353927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cec1cf986564f5ca8e9b9c4187d4da7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.354658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cec1cf986564f5ca8e9b9c4187d4da7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.355353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cec1cf986564f5ca8e9b9c4187d4da7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.363600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cec1cf986564f5ca8e9b9c4187d4da7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.371825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cec1cf986564f5ca8e9b9c4187d4da7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.392531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1134/2245] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:26 2022] DELETE /identity/v3/users/2cec1cf986564f5ca8e9b9c4187d4da7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:26.397116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:26.397408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:26.397689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:26.416880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:26.418127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:26.483217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1112/2246] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:26.493426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:26.494860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:26.505388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:26.554737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e66b69-6917-4966-a841-f9bd7af97056 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:26.558411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:26.559056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:26.559338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:26.559612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:26.568804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:26.571005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:26.580357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1135/2247] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:26.866847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:26.868902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:26.884397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1db8-74b0-4811-aadd-0be1797fd056 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:26.897120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:26.897933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:26.898792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:26.899029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] PATH_INFO: `/v3/projects/48a5ae63688a455191d02cbdb7a416a7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:26.905250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a5ae63688a455191d02cbdb7a416a7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:26.907241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:26.930620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:26.955555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a5ae63688a455191d02cbdb7a416a7', 'request_id': 'req-19a34974-fe98-4b6b-a2e2-3dda6d773aba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.963887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a5ae63688a455191d02cbdb7a416a7', 'request_id': 'req-19a34974-fe98-4b6b-a2e2-3dda6d773aba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.970669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a5ae63688a455191d02cbdb7a416a7', 'request_id': 'req-19a34974-fe98-4b6b-a2e2-3dda6d773aba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.981591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] The token cache is being invalidate because project 48a5ae63688a455191d02cbdb7a416a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:26.981909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.982554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:26.984085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1113/2248] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:26 2022] DELETE /identity/v3/projects/48a5ae63688a455191d02cbdb7a416a7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:26.988062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:26.997905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a556cf-b666-4960-9954-4b8f5f0e707c service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:27.061741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:27.062619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:27.062910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:27.063194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] PATH_INFO: `/v3/users/c8bf188847fa42bd989ed49d2cc8550e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:27.064571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8bf188847fa42bd989ed49d2cc8550e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:27.066565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:27.119857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8bf188847fa42bd989ed49d2cc8550e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.120414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8bf188847fa42bd989ed49d2cc8550e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.121006 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8bf188847fa42bd989ed49d2cc8550e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.129165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8bf188847fa42bd989ed49d2cc8550e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.140693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a58964b7-dd6c-4f9a-8cc5-d6bddfb81fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8bf188847fa42bd989ed49d2cc8550e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.158685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1136/2249] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:26 2022] DELETE /identity/v3/users/c8bf188847fa42bd989ed49d2cc8550e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:27.467106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:27.468579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:27.484676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a34974-fe98-4b6b-a2e2-3dda6d773aba admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:27.568403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:27.569283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:27.569573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:27.570474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] PATH_INFO: `/v3/projects/78347d18859e44a6bfd5e96ae0e339b0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:27.571709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78347d18859e44a6bfd5e96ae0e339b0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:27.573636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:27.636811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78347d18859e44a6bfd5e96ae0e339b0', 'request_id': 'req-0510f456-508f-44ac-bf6c-78cd6a529ad0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.649344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78347d18859e44a6bfd5e96ae0e339b0', 'request_id': 'req-0510f456-508f-44ac-bf6c-78cd6a529ad0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.656977 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78347d18859e44a6bfd5e96ae0e339b0', 'request_id': 'req-0510f456-508f-44ac-bf6c-78cd6a529ad0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.668731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] The token cache is being invalidate because project 78347d18859e44a6bfd5e96ae0e339b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:27.669021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.671014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0510f456-508f-44ac-bf6c-78cd6a529ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:27.671381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1114/2250] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:27 2022] DELETE /identity/v3/projects/78347d18859e44a6bfd5e96ae0e339b0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:27.679363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efaa72-a7bd-447a-a0a5-fbac08b5a804 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:27.680284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efaa72-a7bd-447a-a0a5-fbac08b5a804 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:27.680690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efaa72-a7bd-447a-a0a5-fbac08b5a804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:27.700575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d5efaa72-a7bd-447a-a0a5-fbac08b5a804 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:27.701827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5efaa72-a7bd-447a-a0a5-fbac08b5a804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:27.735283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:27.779662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1137/2251] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:27.793518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057845-2e9d-4cb4-be0e-ac849131342c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:27.793846 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057845-2e9d-4cb4-be0e-ac849131342c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:27.794142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057845-2e9d-4cb4-be0e-ac849131342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:27.817302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cd057845-2e9d-4cb4-be0e-ac849131342c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:27.818552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cd057845-2e9d-4cb4-be0e-ac849131342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:27.826699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1115/2252] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:27.831005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efaa72-a7bd-447a-a0a5-fbac08b5a804 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:27.841615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efaa72-a7bd-447a-a0a5-fbac08b5a804 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:27.846619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:27.847515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:27.847807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:27.848038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:27.852828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:27.853230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:27.855033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:27.880447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1138/2253] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:27.885189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057845-2e9d-4cb4-be0e-ac849131342c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:27.895086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057845-2e9d-4cb4-be0e-ac849131342c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:27.900089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:27.900755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:27.901039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:27.901317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:27.902052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:27.904006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:27.936120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1116/2254] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:27 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:27.940205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:27.950360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65bdf6-5cc6-4824-bc3f-0b3f0f9ac309 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:28.015078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:28.015771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.016035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.016315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.017039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:28.019022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:28.041675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1139/2255] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:27 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:28.047803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:28.058675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151bfa3-de9b-4bcf-a615-9d9836ab6c05 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:28.063460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:28.064140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.064427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.065104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.065432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:28.067074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:28.072975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:28.081886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1117/2256] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:28.084283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:28.096026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7429de3a-a6c4-4392-942a-1bd9ca75f916 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:28.102174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075563b-8e5a-4a79-9912-9e6df190d484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:28.102857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075563b-8e5a-4a79-9912-9e6df190d484 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.103142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075563b-8e5a-4a79-9912-9e6df190d484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.103420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075563b-8e5a-4a79-9912-9e6df190d484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.104147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075563b-8e5a-4a79-9912-9e6df190d484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:28.105845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075563b-8e5a-4a79-9912-9e6df190d484 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:28.116395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1140/2257] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:28.121155 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:28.132013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a788d-d1e9-4121-8236-4669fb35c3c8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:28.139330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:28.140464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.140807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.141111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] PATH_INFO: `/v3/projects/0a755707872f4eba835c9f48f01c217b/users/348dda96484c4ee982e4ac79b667a576/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.157700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a755707872f4eba835c9f48f01c217b, user_id=348dda96484c4ee982e4ac79b667a576, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:28.160381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:28.175506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1118/2258] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:28 2022] PUT /identity/v3/projects/0a755707872f4eba835c9f48f01c217b/users/348dda96484c4ee982e4ac79b667a576/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:28.186183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955dc8f-3345-4a69-9343-ad26584f0df6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.186490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955dc8f-3345-4a69-9343-ad26584f0df6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.186772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955dc8f-3345-4a69-9343-ad26584f0df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.218152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9955dc8f-3345-4a69-9343-ad26584f0df6 None None] MFA Rules not processed for user `348dda96484c4ee982e4ac79b667a576`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:28.219380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9955dc8f-3345-4a69-9343-ad26584f0df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:28.286194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1141/2259] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:28.297811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:28.299574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:28.358164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c8cf8-f4ba-4eb5-9af2-dfe6b1c9cc33 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:28.361776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca869935-2acf-4f16-9be7-f72b474b3e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:28.362423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869935-2acf-4f16-9be7-f72b474b3e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.362705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869935-2acf-4f16-9be7-f72b474b3e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.362981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca869935-2acf-4f16-9be7-f72b474b3e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.371802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca869935-2acf-4f16-9be7-f72b474b3e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:28.373183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca869935-2acf-4f16-9be7-f72b474b3e12 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:28.382911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1119/2260] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:28.404000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955dc8f-3345-4a69-9343-ad26584f0df6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:28.430447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:28.468702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955dc8f-3345-4a69-9343-ad26584f0df6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:28.473532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:28.474436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.474722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.475020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] PATH_INFO: `/v3/users/348dda96484c4ee982e4ac79b667a576` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.476237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] RBAC: Authorizing `identity:delete_user(user_id=348dda96484c4ee982e4ac79b667a576)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:28.478187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:28.526659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348dda96484c4ee982e4ac79b667a576'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:28.527321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348dda96484c4ee982e4ac79b667a576'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:28.527942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348dda96484c4ee982e4ac79b667a576'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:28.536855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348dda96484c4ee982e4ac79b667a576'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:28.545083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348dda96484c4ee982e4ac79b667a576'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:28.565480 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1142/2261] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:28 2022] DELETE /identity/v3/users/348dda96484c4ee982e4ac79b667a576 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:28.570554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.570839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.571113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.589474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:28.590758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:28.665834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1120/2262] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:28.675907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:28.678918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:28.748801 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c958253-c35f-476c-b93f-27508908d91b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:28.752495 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b139379-e12c-40b5-bd73-f202e710bf3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:28.753148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b139379-e12c-40b5-bd73-f202e710bf3e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:28.753431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b139379-e12c-40b5-bd73-f202e710bf3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:28.753708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b139379-e12c-40b5-bd73-f202e710bf3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:28.763931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b139379-e12c-40b5-bd73-f202e710bf3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:28.765404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b139379-e12c-40b5-bd73-f202e710bf3e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:28.775557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1143/2263] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:29.084239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.085917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:29.119966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537da7c-a0a1-453a-bf3d-cd036e4dc6d3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.136595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.139034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.139395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.139599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] PATH_INFO: `/v3/projects/0a755707872f4eba835c9f48f01c217b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.140911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a755707872f4eba835c9f48f01c217b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.143311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.211402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a755707872f4eba835c9f48f01c217b', 'request_id': 'req-251218eb-6913-4bb0-aeca-d50c0bae92b4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:29.219717 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a755707872f4eba835c9f48f01c217b', 'request_id': 'req-251218eb-6913-4bb0-aeca-d50c0bae92b4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:29.234816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a755707872f4eba835c9f48f01c217b', 'request_id': 'req-251218eb-6913-4bb0-aeca-d50c0bae92b4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:29.251429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] The token cache is being invalidate because project 0a755707872f4eba835c9f48f01c217b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:29.251724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:29.252369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-251218eb-6913-4bb0-aeca-d50c0bae92b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:29.257957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1121/2264] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:29 2022] DELETE /identity/v3/projects/0a755707872f4eba835c9f48f01c217b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:29.259628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498981-e23d-4705-a3b5-e083d9aae07d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.260112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498981-e23d-4705-a3b5-e083d9aae07d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.260211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498981-e23d-4705-a3b5-e083d9aae07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.280958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f9498981-e23d-4705-a3b5-e083d9aae07d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:29.282398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9498981-e23d-4705-a3b5-e083d9aae07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:29.335811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:29.357012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1144/2265] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:29.366321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079150a0-f1cf-4ee4-8a6d-2b4fa0bd2fe0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.366636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079150a0-f1cf-4ee4-8a6d-2b4fa0bd2fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.366983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079150a0-f1cf-4ee4-8a6d-2b4fa0bd2fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.373872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:29.390107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-079150a0-f1cf-4ee4-8a6d-2b4fa0bd2fe0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:29.391349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-079150a0-f1cf-4ee4-8a6d-2b4fa0bd2fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:29.400016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1122/2266] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:29.404273 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9498981-e23d-4705-a3b5-e083d9aae07d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.414310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9498981-e23d-4705-a3b5-e083d9aae07d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.418877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.419568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.419854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.420127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.420851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:29.421158 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.422828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.435584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1145/2267] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:29.440227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079150a0-f1cf-4ee4-8a6d-2b4fa0bd2fe0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.451400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079150a0-f1cf-4ee4-8a6d-2b4fa0bd2fe0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.456933 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.457597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.457995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.458204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.458932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.461579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.492947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1123/2268] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:29 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:29.497190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.507937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae085a19-10ba-4037-ad1e-a3dd032b99ca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.563366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.564032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.564316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.564589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.565278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.567235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.590872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1146/2269] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:29 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:29.592330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.603421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e0986-d155-445d-8efe-b81670c45619 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.607981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.608631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.608914 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.609192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.609939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.611532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.622039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1124/2270] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:29.625994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.635816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e74268-fed5-4622-b276-74d4ef45b128 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.640851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19d913-a671-444a-89df-b8dd5e8a4e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.641513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d913-a671-444a-89df-b8dd5e8a4e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.641829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d913-a671-444a-89df-b8dd5e8a4e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.642112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19d913-a671-444a-89df-b8dd5e8a4e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.642844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19d913-a671-444a-89df-b8dd5e8a4e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.644508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19d913-a671-444a-89df-b8dd5e8a4e67 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.655263 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1147/2271] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:29.659429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.670416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e61de-272a-45a6-ac35-1b1c767175a6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.675329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.676312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.676552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.676835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] PATH_INFO: `/v3/projects/731df982fd964de6bc4545103f7e1047/users/6e097835e6604a02a38b291786033f06/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.691896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731df982fd964de6bc4545103f7e1047, user_id=6e097835e6604a02a38b291786033f06, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.694592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.707915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1125/2272] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:29 2022] PUT /identity/v3/projects/731df982fd964de6bc4545103f7e1047/users/6e097835e6604a02a38b291786033f06/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:29.713651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.713984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.714274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.733303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:29.734628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:29.796026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1148/2273] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:29.805803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:29.807843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:29.877585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7458ee-361e-4598-b295-080e46847a17 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:29.881366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949cf88-ddaf-4dfd-b962-f317eac7b7ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:29.882045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949cf88-ddaf-4dfd-b962-f317eac7b7ea service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:29.882834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949cf88-ddaf-4dfd-b962-f317eac7b7ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:29.883115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949cf88-ddaf-4dfd-b962-f317eac7b7ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:29.896107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949cf88-ddaf-4dfd-b962-f317eac7b7ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:29.897554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949cf88-ddaf-4dfd-b962-f317eac7b7ea service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:29.908288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1126/2274] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:31.590806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:31.594882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:31.622317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0dd67-680f-4e5a-955c-fbda4bf2f5bb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:31.639604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:31.640833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:31.641326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:31.641828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] PATH_INFO: `/v3/users/e7f0b4ce977e46b28f8aeec456659bea` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:31.643516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f0b4ce977e46b28f8aeec456659bea)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:31.646910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:31.739416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f0b4ce977e46b28f8aeec456659bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:31.740365 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f0b4ce977e46b28f8aeec456659bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:31.742203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f0b4ce977e46b28f8aeec456659bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:31.745186 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:31.759894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f0b4ce977e46b28f8aeec456659bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:31.767976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f0b4ce977e46b28f8aeec456659bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:31.798723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1149/2275] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:31 2022] DELETE /identity/v3/users/e7f0b4ce977e46b28f8aeec456659bea => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:31.801080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:31.801586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:31.802335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:31.821938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:31.823953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:31.899783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1127/2276] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:15:31.909939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:31.911404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:31.976626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc84a8-b5be-42a0-822b-4cf01480b31b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:31.980261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:31.980941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:31.981204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:31.981480 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:31.991603 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:31.993226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:32.004117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1150/2277] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:32.310792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:32.312711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:32.346074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e8b5-9681-47ff-b004-3f90a47e0b9d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:32.354453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:32.355502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:32.355862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:32.356279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] PATH_INFO: `/v3/projects/2c7e084cb0c4475796ef2160db9291b9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:32.357820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7e084cb0c4475796ef2160db9291b9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:32.359873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:32.410718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:32.433885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7e084cb0c4475796ef2160db9291b9', 'request_id': 'req-c4d21207-912e-4255-9f06-157680486c86'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.443336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7e084cb0c4475796ef2160db9291b9', 'request_id': 'req-c4d21207-912e-4255-9f06-157680486c86'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.457373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7e084cb0c4475796ef2160db9291b9', 'request_id': 'req-c4d21207-912e-4255-9f06-157680486c86'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.471148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] The token cache is being invalidate because project 2c7e084cb0c4475796ef2160db9291b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:32.471516 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.472709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.478563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1128/2278] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:32 2022] DELETE /identity/v3/projects/2c7e084cb0c4475796ef2160db9291b9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:32.479828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:32.494687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38540bb-bb4f-4235-8553-a7dbb3b0e727 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:32.640692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:32.642540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:32.643017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:32.643654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] PATH_INFO: `/v3/users/7efa3659caa64551b2196069ff023f04` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:32.645511 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7efa3659caa64551b2196069ff023f04)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:32.648437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:32.739295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7efa3659caa64551b2196069ff023f04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.740427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7efa3659caa64551b2196069ff023f04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.741249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7efa3659caa64551b2196069ff023f04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.751506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7efa3659caa64551b2196069ff023f04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.762573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ddc34cb-23d2-495b-9313-3a12b3733839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7efa3659caa64551b2196069ff023f04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:32.771906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:32.801753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1151/2279] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:32 2022] DELETE /identity/v3/users/7efa3659caa64551b2196069ff023f04 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:33.555692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:33.557932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:33.586107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d21207-912e-4255-9f06-157680486c86 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:33.686273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:33.687511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:33.688000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:33.688489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] PATH_INFO: `/v3/projects/2361dfc7918649c49a4a8a6a034c4100` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:33.690401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2361dfc7918649c49a4a8a6a034c4100)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:33.692537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:33.776363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2361dfc7918649c49a4a8a6a034c4100', 'request_id': 'req-105a8a79-9699-425f-ba3b-10a6724f8362'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:33.788991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2361dfc7918649c49a4a8a6a034c4100', 'request_id': 'req-105a8a79-9699-425f-ba3b-10a6724f8362'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:33.799855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2361dfc7918649c49a4a8a6a034c4100', 'request_id': 'req-105a8a79-9699-425f-ba3b-10a6724f8362'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:33.814680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] The token cache is being invalidate because project 2361dfc7918649c49a4a8a6a034c4100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:33.815053 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:33.815642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105a8a79-9699-425f-ba3b-10a6724f8362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:33.817888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1129/2280] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:33 2022] DELETE /identity/v3/projects/2361dfc7918649c49a4a8a6a034c4100 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:33.823496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d00a-1a80-4b39-856f-070d8bc9d311 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:33.824008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d00a-1a80-4b39-856f-070d8bc9d311 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:33.824106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77d00a-1a80-4b39-856f-070d8bc9d311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:33.845440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5f77d00a-1a80-4b39-856f-070d8bc9d311 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:33.846841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f77d00a-1a80-4b39-856f-070d8bc9d311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:33.928977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1152/2281] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:15:33.937211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21320e-89b5-45e5-966a-e6aea30e2383 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:33.937589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21320e-89b5-45e5-966a-e6aea30e2383 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:33.938015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21320e-89b5-45e5-966a-e6aea30e2383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:33.961599 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cb21320e-89b5-45e5-966a-e6aea30e2383 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:33.962730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cb21320e-89b5-45e5-966a-e6aea30e2383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:33.978391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77d00a-1a80-4b39-856f-070d8bc9d311 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:33.988276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1130/2282] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:34.002226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77d00a-1a80-4b39-856f-070d8bc9d311 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.012839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.014009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.014560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.015196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.017869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:34.018544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.021010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.044853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1153/2283] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:15:34.048010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21320e-89b5-45e5-966a-e6aea30e2383 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.061203 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21320e-89b5-45e5-966a-e6aea30e2383 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.066528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.067298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.067673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.068022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.068790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.071842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.105837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1131/2284] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:34 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:34.109544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.123686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2df74-6467-46df-9d2b-932230b338a0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.195494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.196372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.197862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.197862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.198086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.200127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.225506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1154/2285] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:34 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:34.230422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.249413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6dfd8-eea5-4794-b60f-cf95adbc77ec admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.257279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.258503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.258787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.259060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.259883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.261542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.275770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1132/2286] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:34.280230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.293142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c310c-ac2f-4a26-9679-c61ff8470b94 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.299415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ada1b-5a80-4b37-8278-0affc7887e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.300314 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ada1b-5a80-4b37-8278-0affc7887e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.300666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ada1b-5a80-4b37-8278-0affc7887e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.300950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ada1b-5a80-4b37-8278-0affc7887e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.302484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ada1b-5a80-4b37-8278-0affc7887e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.303804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ada1b-5a80-4b37-8278-0affc7887e02 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.321955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1155/2287] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:34.322814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.338754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc859c-e51d-451a-b36e-db828f3d60d4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.346923 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.348271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.348655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.348997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] PATH_INFO: `/v3/projects/cc69c0f9eab9408b854b6d6f5275d7ba/users/c2bd24cb8173473caeba87d9b989196f/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.369221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:34.374804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc69c0f9eab9408b854b6d6f5275d7ba, user_id=c2bd24cb8173473caeba87d9b989196f, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.380075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.399981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1133/2288] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:34 2022] PUT /identity/v3/projects/cc69c0f9eab9408b854b6d6f5275d7ba/users/c2bd24cb8173473caeba87d9b989196f/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:34.411283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b2ba-35a8-4191-91ef-d31b90eda42b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.412344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b2ba-35a8-4191-91ef-d31b90eda42b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.412772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43b2ba-35a8-4191-91ef-d31b90eda42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.447799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c43b2ba-35a8-4191-91ef-d31b90eda42b None None] MFA Rules not processed for user `c2bd24cb8173473caeba87d9b989196f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:34.449042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c43b2ba-35a8-4191-91ef-d31b90eda42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:34.506477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:34.566436 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1156/2289] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:34.571593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.687486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7f08e-0fa6-4d66-8373-fb8aa7688a1c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.692413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.693079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.693364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.693644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.694380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.696839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.732307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1134/2290] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:34 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:34.733612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43b2ba-35a8-4191-91ef-d31b90eda42b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.745819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43b2ba-35a8-4191-91ef-d31b90eda42b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.828272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.830102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.830571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.830983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.831864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.834061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.883721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1157/2291] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:34 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:34.887841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.903994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ef620-f677-40c1-82ac-236d00f579b2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.911746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.912438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.912728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.913011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.913956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.915600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.927313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1135/2292] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:15:34.932248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:34.943615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50543db1-9208-4371-a654-16ac362827bc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:34.948972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:34.949931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:34.950223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:34.950505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] PATH_INFO: `/v3/projects/848bf6d05c33472187f9f75cdc9550f2/users/3f34bd66019347a0b0ceebb449d7bca2/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:34.971235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] RBAC: Authorizing `identity:create_grant(project_id=848bf6d05c33472187f9f75cdc9550f2, user_id=3f34bd66019347a0b0ceebb449d7bca2, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:34.976575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:34.993017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1158/2293] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:34 2022] PUT /identity/v3/projects/848bf6d05c33472187f9f75cdc9550f2/users/3f34bd66019347a0b0ceebb449d7bca2/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:34.997798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:35.013143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2354e0a-ec81-495c-82c1-57402750bff0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:35.085083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fa0db-3c88-4104-ad4d-c5144faa54a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:35.086337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fa0db-3c88-4104-ad4d-c5144faa54a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:35.086658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fa0db-3c88-4104-ad4d-c5144faa54a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:35.086913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fa0db-3c88-4104-ad4d-c5144faa54a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:35.089012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fa0db-3c88-4104-ad4d-c5144faa54a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:35.090390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fa0db-3c88-4104-ad4d-c5144faa54a3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:35.106226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1136/2294] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:35.107805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:35.120028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db51a70-0d8e-4c3d-8d2d-2c49cd92145c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:35.124870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:35.125925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:35.126830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:35.127185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] PATH_INFO: `/v3/projects/848bf6d05c33472187f9f75cdc9550f2/users/3f34bd66019347a0b0ceebb449d7bca2/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:35.129762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=848bf6d05c33472187f9f75cdc9550f2, user_id=3f34bd66019347a0b0ceebb449d7bca2, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:35.133301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:35.157933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1159/2295] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:35 2022] PUT /identity/v3/projects/848bf6d05c33472187f9f75cdc9550f2/users/3f34bd66019347a0b0ceebb449d7bca2/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:35.160308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111c5bf-5529-42d0-85cb-6f2735b6771d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:35.161219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111c5bf-5529-42d0-85cb-6f2735b6771d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:35.161552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111c5bf-5529-42d0-85cb-6f2735b6771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:35.196798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9111c5bf-5529-42d0-85cb-6f2735b6771d None None] MFA Rules not processed for user `3f34bd66019347a0b0ceebb449d7bca2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:35.199630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9111c5bf-5529-42d0-85cb-6f2735b6771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:35.263911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:35.315232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1137/2296] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:35.327545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:35.329997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:35.423269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028fb9c-152c-4792-8494-21a8439f5702 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:35.427835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43c209-ab65-4301-a86b-def80ca1720d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:35.428571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c209-ab65-4301-a86b-def80ca1720d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:35.428872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c209-ab65-4301-a86b-def80ca1720d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:35.429907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43c209-ab65-4301-a86b-def80ca1720d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:35.441339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c209-ab65-4301-a86b-def80ca1720d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:35.443012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43c209-ab65-4301-a86b-def80ca1720d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:35.455832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1160/2297] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:35.559147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111c5bf-5529-42d0-85cb-6f2735b6771d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:35.712750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111c5bf-5529-42d0-85cb-6f2735b6771d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:35.719335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:35.720258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:35.720542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:35.720877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] PATH_INFO: `/v3/users/c2bd24cb8173473caeba87d9b989196f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:35.724139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2bd24cb8173473caeba87d9b989196f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:35.725843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:35.815469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2bd24cb8173473caeba87d9b989196f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:35.816254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2bd24cb8173473caeba87d9b989196f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:35.829543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2bd24cb8173473caeba87d9b989196f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:35.830346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2bd24cb8173473caeba87d9b989196f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:35.842404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2bd24cb8173473caeba87d9b989196f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:35.865012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1138/2298] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:35 2022] DELETE /identity/v3/users/c2bd24cb8173473caeba87d9b989196f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:35.872638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:35.874013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:35.874832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:35.914021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:35.916294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:35.973379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:36.034229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1161/2299] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:36.063937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:36.066058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:36.183904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb68849-3fdb-4301-8a04-4b64cfacba24 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:36.189703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:36.190409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:36.190701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:36.191035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:36.227951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:36.229442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:36.242311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1139/2300] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:36.643823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:36.645373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:36.666781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037bd48-928b-47e9-ae9f-173b9df741ac None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:36.673757 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:36.675234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:36.675675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:36.676109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] PATH_INFO: `/v3/projects/cc69c0f9eab9408b854b6d6f5275d7ba` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:36.684895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc69c0f9eab9408b854b6d6f5275d7ba)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:36.688874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:36.755303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc69c0f9eab9408b854b6d6f5275d7ba', 'request_id': 'req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.767700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc69c0f9eab9408b854b6d6f5275d7ba', 'request_id': 'req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.779290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc69c0f9eab9408b854b6d6f5275d7ba', 'request_id': 'req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.795089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] The token cache is being invalidate because project cc69c0f9eab9408b854b6d6f5275d7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:36.795409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.796774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e53b7a36-cb6e-4d69-bceb-5c96ea0597b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.803593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1162/2301] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:36 2022] DELETE /identity/v3/projects/cc69c0f9eab9408b854b6d6f5275d7ba => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:36.804797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:36.817621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137266a3-4e6b-42e4-b7cc-721ecdc1e1ed service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:36.844762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:36.914126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:36.915166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:36.915443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:36.915721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] PATH_INFO: `/v3/users/3f34bd66019347a0b0ceebb449d7bca2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:36.917060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f34bd66019347a0b0ceebb449d7bca2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:36.919225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:36.981425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f34bd66019347a0b0ceebb449d7bca2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.982176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f34bd66019347a0b0ceebb449d7bca2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.995597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f34bd66019347a0b0ceebb449d7bca2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:36.996593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f34bd66019347a0b0ceebb449d7bca2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:37.015378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f34bd66019347a0b0ceebb449d7bca2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:37.057193 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1140/2302] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:36 2022] DELETE /identity/v3/users/3f34bd66019347a0b0ceebb449d7bca2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:37.098152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbfa26-af30-4cd2-a522-d533514bf148 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.101509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbfa26-af30-4cd2-a522-d533514bf148 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.102347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbfa26-af30-4cd2-a522-d533514bf148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.183156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3bcbfa26-af30-4cd2-a522-d533514bf148 None None] MFA Rules not processed for user `6e097835e6604a02a38b291786033f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:37.185373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3bcbfa26-af30-4cd2-a522-d533514bf148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:37.311672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:37.313411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:37.315133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1163/2303] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:37.434058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e817d-52ae-417b-8854-ed3cae08074b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:37.443180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:37.444344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.444899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.445413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.447277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:37.451264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:37.502517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1141/2304] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:37 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:37.506758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbfa26-af30-4cd2-a522-d533514bf148 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:37.520395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbfa26-af30-4cd2-a522-d533514bf148 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:37.583813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:37.585651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:37.618144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:37.660205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:37.692453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:37.693707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.694246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.695424 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.697471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:37.700735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:37.730686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1164/2305] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:37 2022] POST /identity/v3/users => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:15:37.735228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:37.747491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49208741-422b-46f5-8b6d-f47397420b26 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:37.753336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:37.754342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.754731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a1eba-366d-431c-8b68-efc968d4e4ff admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:37.754848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.755563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.755968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:37.757690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:37.765857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:37.766965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.767334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.767678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] PATH_INFO: `/v3/projects/848bf6d05c33472187f9f75cdc9550f2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.769191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=848bf6d05c33472187f9f75cdc9550f2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:37.773336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:37.775367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1165/2306] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:15:37.779209 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:37.790397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724746c4-e901-463d-bde8-cf1b02dd3ecd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:37.803285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:37.804279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.804653 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.804980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] PATH_INFO: `/v3/projects/4ee4194f9bfb4acb93f9b0c48e270202/users/57015b7bef2a478687ca669af612fcaf/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.823535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee4194f9bfb4acb93f9b0c48e270202, user_id=57015b7bef2a478687ca669af612fcaf, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:37.827473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:37.840333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848bf6d05c33472187f9f75cdc9550f2', 'request_id': 'req-8b30beb8-47bc-4d34-839a-e3e13f308896'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:37.850057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848bf6d05c33472187f9f75cdc9550f2', 'request_id': 'req-8b30beb8-47bc-4d34-839a-e3e13f308896'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:37.850191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1166/2307] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:37 2022] PUT /identity/v3/projects/4ee4194f9bfb4acb93f9b0c48e270202/users/57015b7bef2a478687ca669af612fcaf/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:37.854290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:37.858278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848bf6d05c33472187f9f75cdc9550f2', 'request_id': 'req-8b30beb8-47bc-4d34-839a-e3e13f308896'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:37.870659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] The token cache is being invalidate because project 848bf6d05c33472187f9f75cdc9550f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:37.871547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:37.872168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8b30beb8-47bc-4d34-839a-e3e13f308896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:37.874089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1142/2308] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:37 2022] DELETE /identity/v3/projects/848bf6d05c33472187f9f75cdc9550f2 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:37.879314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea5c83-5276-4fd7-aef7-cf165ca8f7c8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.879751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea5c83-5276-4fd7-aef7-cf165ca8f7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.880135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea5c83-5276-4fd7-aef7-cf165ca8f7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.903808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ceea5c83-5276-4fd7-aef7-cf165ca8f7c8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:37.905036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ceea5c83-5276-4fd7-aef7-cf165ca8f7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:37.936571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a1aa5-a0d3-4a37-9c12-3e8a32bc18f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:37.942072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:37.942805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.943135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.943418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.944179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:37.946201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:37.958434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1167/2309] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:15:37.963457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:37.965768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:37.981906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca403df-e2e2-4140-9db4-b798180398b8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:37.989523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d68ae-7c92-4787-8001-043addbd6995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:37.990826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d68ae-7c92-4787-8001-043addbd6995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:37.991112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d68ae-7c92-4787-8001-043addbd6995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:37.991915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d68ae-7c92-4787-8001-043addbd6995 admin admin] PATH_INFO: `/v3/projects/4ee4194f9bfb4acb93f9b0c48e270202/users/57015b7bef2a478687ca669af612fcaf/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:37.994876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d68ae-7c92-4787-8001-043addbd6995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee4194f9bfb4acb93f9b0c48e270202, user_id=57015b7bef2a478687ca669af612fcaf, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:37.998324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d68ae-7c92-4787-8001-043addbd6995 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:38.013880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1168/2310] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:37 2022] PUT /identity/v3/projects/4ee4194f9bfb4acb93f9b0c48e270202/users/57015b7bef2a478687ca669af612fcaf/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:38.032671 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1143/2311] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:38.040784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeba934-623c-486e-864c-2a4134a14584 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.041068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeba934-623c-486e-864c-2a4134a14584 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.041441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeba934-623c-486e-864c-2a4134a14584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.064502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6eeba934-623c-486e-864c-2a4134a14584 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:38.065740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6eeba934-623c-486e-864c-2a4134a14584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:38.148251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1169/2312] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:38.152300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea5c83-5276-4fd7-aef7-cf165ca8f7c8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:38.163153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea5c83-5276-4fd7-aef7-cf165ca8f7c8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:38.168799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:38.169869 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.170002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.170239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.171291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:15:38.171641 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:38.173591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:38.189976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1144/2313] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:15:38.195222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeba934-623c-486e-864c-2a4134a14584 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:38.206462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeba934-623c-486e-864c-2a4134a14584 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:38.211398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:38.214251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.214251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.214602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.215403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:38.218120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:38.260039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1170/2314] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:15:38 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:38.262370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:38.275940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29796882-c247-49fd-8451-a539446abb27 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:38.310974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:38.377659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:38.378379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.378708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.378983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.379676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:38.381604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:38.427628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1145/2315] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:15:38 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:15:38.428128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:38.443491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06fd19-c9b9-4033-918e-5605cfac0f42 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:38.457750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:38.466024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.469470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.469878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.470762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:38.472768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:38.488598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:38.547921 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1171/2316] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:38.549286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:38.571177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80eca5-4d15-46c4-ba28-1eac854e4426 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:38.586739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15820397-3886-49fd-9724-3517eab48418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:38.587703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15820397-3886-49fd-9724-3517eab48418 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.592032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15820397-3886-49fd-9724-3517eab48418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.592535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15820397-3886-49fd-9724-3517eab48418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.619795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15820397-3886-49fd-9724-3517eab48418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:38.621748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15820397-3886-49fd-9724-3517eab48418 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:38.671194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:38.681049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1146/2317] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:15:38.698762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd244ca3-fa0c-4f1b-8228-34a8dcc228d2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:38.726492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:38.727609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.727899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.728181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] PATH_INFO: `/v3/projects/612b1a0e56de4187a0d051452e58b4da/users/be5e27c4932c4583935b2afafc290e2d/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.767342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=612b1a0e56de4187a0d051452e58b4da, user_id=be5e27c4932c4583935b2afafc290e2d, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:38.776488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:38.805264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1172/2318] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:15:38 2022] PUT /identity/v3/projects/612b1a0e56de4187a0d051452e58b4da/users/be5e27c4932c4583935b2afafc290e2d/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:15:38.819463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adb958-0b2d-4013-975d-50eaa6842abc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:38.819954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adb958-0b2d-4013-975d-50eaa6842abc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:38.820397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adb958-0b2d-4013-975d-50eaa6842abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:38.862940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d1adb958-0b2d-4013-975d-50eaa6842abc None None] MFA Rules not processed for user `be5e27c4932c4583935b2afafc290e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:38.865266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d1adb958-0b2d-4013-975d-50eaa6842abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:39.018655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1147/2319] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:39.035699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:39.037622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:39.210169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4362e3-8202-4579-99ce-54e7cc8a8c1d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:39.227650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5af4fa-7b55-461e-8152-5de39f5dcd6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:39.231509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af4fa-7b55-461e-8152-5de39f5dcd6c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:39.232020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af4fa-7b55-461e-8152-5de39f5dcd6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:39.233098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af4fa-7b55-461e-8152-5de39f5dcd6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:39.255980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5af4fa-7b55-461e-8152-5de39f5dcd6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:39.261427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5af4fa-7b55-461e-8152-5de39f5dcd6c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:39.318529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1173/2320] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:39.336407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adb958-0b2d-4013-975d-50eaa6842abc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:39.477165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adb958-0b2d-4013-975d-50eaa6842abc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:39.494830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:39.495711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:39.496016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:39.496427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] PATH_INFO: `/v3/users/be5e27c4932c4583935b2afafc290e2d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:39.497475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5e27c4932c4583935b2afafc290e2d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:39.499498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:39.627041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:39.637104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5e27c4932c4583935b2afafc290e2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:39.639768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5e27c4932c4583935b2afafc290e2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:39.656870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5e27c4932c4583935b2afafc290e2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:39.657756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5e27c4932c4583935b2afafc290e2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:39.678015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5e27c4932c4583935b2afafc290e2d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:39.758963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1148/2321] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:39 2022] DELETE /identity/v3/users/be5e27c4932c4583935b2afafc290e2d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:39.763605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:39.764558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:39.764558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:39.829408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:39.830925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:39.929515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1174/2322] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:39.941831 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:39.943643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:40.059331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3e827-065b-4899-ad15-b8117c8b3ec1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:40.065255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:40.066757 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:40.067169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:40.067529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:40.106233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:40.108048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:40.129699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1149/2323] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:40.705782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:40.708739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:40.730359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e80d19-b2e6-4252-8ba4-efe8f2ac9cd8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:40.754187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:40.755962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:40.756514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:40.757005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] PATH_INFO: `/v3/projects/612b1a0e56de4187a0d051452e58b4da` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:40.758508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=612b1a0e56de4187a0d051452e58b4da)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:40.760695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:40.862272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612b1a0e56de4187a0d051452e58b4da', 'request_id': 'req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:40.874019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612b1a0e56de4187a0d051452e58b4da', 'request_id': 'req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:40.887031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612b1a0e56de4187a0d051452e58b4da', 'request_id': 'req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:40.896770 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:40.903053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] The token cache is being invalidate because project 612b1a0e56de4187a0d051452e58b4da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:40.903704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:40.904489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a2d0986-cb9e-40b9-9bfb-0697fc0855b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:40.907053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1175/2324] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:40 2022] DELETE /identity/v3/projects/612b1a0e56de4187a0d051452e58b4da => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:41.909867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:41.912972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:42.051531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b0cb-2479-4cd1-b0b8-e9844af86216 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:42.105465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:42.146683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db4d5d-0ec0-47ea-a0da-2334878fb599 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:42.146683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db4d5d-0ec0-47ea-a0da-2334878fb599 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:42.146683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db4d5d-0ec0-47ea-a0da-2334878fb599 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:42.147044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db4d5d-0ec0-47ea-a0da-2334878fb599 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:42.156974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-74db4d5d-0ec0-47ea-a0da-2334878fb599 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:42.174574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db4d5d-0ec0-47ea-a0da-2334878fb599 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:42.174836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db4d5d-0ec0-47ea-a0da-2334878fb599 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:42.187529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1150/2325] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:44.035269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:44.035747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:44.036015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:44.071478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] MFA Rules not processed for user `57015b7bef2a478687ca669af612fcaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:44.073252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:44.133187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90018541-6cb7-41d9-a616-be5fa5bade5b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:44.133776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90018541-6cb7-41d9-a616-be5fa5bade5b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:44.133927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90018541-6cb7-41d9-a616-be5fa5bade5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:44.160399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-90018541-6cb7-41d9-a616-be5fa5bade5b None None] MFA Rules not processed for user `16b8389777744e6581ad14be85731f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:44.162605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-90018541-6cb7-41d9-a616-be5fa5bade5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:44.223686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1176/2326] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:15:44.241542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:44.245351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:44.259889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1151/2327] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:15:44.351863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46929e-78cd-42dc-b409-9d1e30e47185 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:44.355755 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:44.356426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:44.356718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:44.357005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:44.358920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:44.438091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:44.439582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:44.467224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1177/2328] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:44.707835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceeb5c8-1ee3-40ff-a5a2-61e994cc3ed7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:44.708131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceeb5c8-1ee3-40ff-a5a2-61e994cc3ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:44.708403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceeb5c8-1ee3-40ff-a5a2-61e994cc3ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:44.822149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cceeb5c8-1ee3-40ff-a5a2-61e994cc3ed7 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:44.824138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cceeb5c8-1ee3-40ff-a5a2-61e994cc3ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:44.919444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1152/2329] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:44.972627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:44.983999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:45.016970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:45.137007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a18132-9c0b-4f41-b373-e302de83f064 service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:45.153183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:45.154154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:45.154523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:45.154878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:45.166139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:45.168742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:45.183738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1178/2330] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:15:49.112321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c6de-f881-4498-8a7c-3d20e974afaa None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:49.113017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c6de-f881-4498-8a7c-3d20e974afaa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:49.113017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c6de-f881-4498-8a7c-3d20e974afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:49.173202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2b55c6de-f881-4498-8a7c-3d20e974afaa None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:49.174612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2b55c6de-f881-4498-8a7c-3d20e974afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:49.182295 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1153/2331] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:49.196143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:49.209358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed425b4c-6fcf-4946-82a0-2231cdd85b17 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:49.220107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:49.220786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:49.221077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:49.221363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:49.232002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:49.233663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:49.247033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1179/2332] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:15:49.938462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130072-5c7a-4701-a488-98f890f230ee None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:49.938763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130072-5c7a-4701-a488-98f890f230ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:49.939082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130072-5c7a-4701-a488-98f890f230ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:49.959791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6c130072-5c7a-4701-a488-98f890f230ee None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:49.961550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6c130072-5c7a-4701-a488-98f890f230ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:49.969555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1154/2333] 10.222.0.88 () {58 vars in 967 bytes} [Thu Jun 16 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:49.984922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:49.997950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a501ba-149d-410b-99ae-bb6350edc321 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:50.004111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:50.004990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:50.005278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:50.005557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:50.018622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:50.020665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:50.032001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1180/2334] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:15:50.453690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f26741-0696-40d4-9d2c-9e0d6d5f6bc2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:50.454620 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f26741-0696-40d4-9d2c-9e0d6d5f6bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:50.455077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f26741-0696-40d4-9d2c-9e0d6d5f6bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:50.474798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-10f26741-0696-40d4-9d2c-9e0d6d5f6bc2 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:50.476346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-10f26741-0696-40d4-9d2c-9e0d6d5f6bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:50.485377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1155/2335] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:15:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:50.508328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:50.526936 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e5534-831b-4548-ab78-e8d3ce8c4854 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:50.556634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:50.557310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:50.557602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:50.557916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:50.579808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:50.581462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:50.599572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1181/2336] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:15:51.100374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c60cfa-2171-4f85-9949-eb7f5c760a74 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:51.100715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c60cfa-2171-4f85-9949-eb7f5c760a74 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:51.101021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c60cfa-2171-4f85-9949-eb7f5c760a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:51.151667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-98c60cfa-2171-4f85-9949-eb7f5c760a74 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:51.153346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-98c60cfa-2171-4f85-9949-eb7f5c760a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:51.162082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1156/2337] 10.222.0.88 () {58 vars in 967 bytes} [Thu Jun 16 16:15:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:51.176024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:51.188467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c6fa1-7609-46f6-b9e4-1048ea9e4c33 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:51.195153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f615750-9180-489a-8808-39033d90904b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:51.195914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f615750-9180-489a-8808-39033d90904b service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:51.196277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f615750-9180-489a-8808-39033d90904b service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:51.196632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f615750-9180-489a-8808-39033d90904b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:51.213664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f615750-9180-489a-8808-39033d90904b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:51.215418 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f615750-9180-489a-8808-39033d90904b service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:51.226823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1182/2338] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:15:56.000370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d2533-60c8-4b4a-aae3-f92e81523e49 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:56.000986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d2533-60c8-4b4a-aae3-f92e81523e49 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:56.000986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d2533-60c8-4b4a-aae3-f92e81523e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:56.020917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8b7d2533-60c8-4b4a-aae3-f92e81523e49 None None] MFA Rules not processed for user `6e097835e6604a02a38b291786033f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:56.022941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8b7d2533-60c8-4b4a-aae3-f92e81523e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:56.030882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1157/2339] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:59.063263 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f615750-9180-489a-8808-39033d90904b service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:59.065140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8f615750-9180-489a-8808-39033d90904b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.150090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f615750-9180-489a-8808-39033d90904b service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:59.156033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:59.156925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.157218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.157502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] PATH_INFO: `/v3/users/6e0dcd599835413a9271a038bec143a1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.159642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0dcd599835413a9271a038bec143a1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:59.161596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:59.202566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:59.211904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0dcd599835413a9271a038bec143a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.212553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0dcd599835413a9271a038bec143a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.213184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0dcd599835413a9271a038bec143a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.221962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0dcd599835413a9271a038bec143a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.229532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0dcd599835413a9271a038bec143a1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.248098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1183/2340] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:15:59 2022] DELETE /identity/v3/users/6e0dcd599835413a9271a038bec143a1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:15:59.252689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.253165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.253254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.272626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:15:59.273887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.277987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:15:59.337752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1158/2341] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:15:59.351900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:59.353415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.371510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:59.372895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.422040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64096c-adff-46ad-9376-379ccb9920b1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:59.426679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:59.427426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.427803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.428333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.435891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2124aa5-c7c4-40a9-98be-6b1ebc0de2cd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:59.440227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558a28e-7c79-4807-977d-677c938de24d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:59.440698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:59.440873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a28e-7c79-4807-977d-677c938de24d service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.441338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a28e-7c79-4807-977d-677c938de24d service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.441616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a28e-7c79-4807-977d-677c938de24d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.442570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:59.443721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a558a28e-7c79-4807-977d-677c938de24d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.454509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1184/2342] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:15:59.517784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558a28e-7c79-4807-977d-677c938de24d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:59.519337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558a28e-7c79-4807-977d-677c938de24d service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:59.530447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1159/2343] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:15:59 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:15:59.651098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:15:59.652781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:15:59.668919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79481f-3370-4462-a0c1-d2660ce73c38 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:15:59.674485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:15:59.676397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:15:59.676696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:15:59.676983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] PATH_INFO: `/v3/projects/999705a6bba24e4198240e1f8b08c4d9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:15:59.678421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999705a6bba24e4198240e1f8b08c4d9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:15:59.680428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:15:59.732096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999705a6bba24e4198240e1f8b08c4d9', 'request_id': 'req-19f248a6-cd15-4dcd-b574-049fca0585f0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.741145 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999705a6bba24e4198240e1f8b08c4d9', 'request_id': 'req-19f248a6-cd15-4dcd-b574-049fca0585f0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.748282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999705a6bba24e4198240e1f8b08c4d9', 'request_id': 'req-19f248a6-cd15-4dcd-b574-049fca0585f0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.760323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] The token cache is being invalidate because project 999705a6bba24e4198240e1f8b08c4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:15:59.760616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.761263 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:15:59.762883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1185/2344] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:15:59 2022] DELETE /identity/v3/projects/999705a6bba24e4198240e1f8b08c4d9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:16:04.129578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013514bd-f6c0-4779-9ab4-f807310ea832 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:04.130122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013514bd-f6c0-4779-9ab4-f807310ea832 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:04.130240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013514bd-f6c0-4779-9ab4-f807310ea832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:04.151882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-013514bd-f6c0-4779-9ab4-f807310ea832 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:04.153094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-013514bd-f6c0-4779-9ab4-f807310ea832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:04.211673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1160/2345] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:04.224877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:16:04.227076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:04.274789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:04.305044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f248a6-cd15-4dcd-b574-049fca0585f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:16:04.311661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:16:04.312326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:04.312616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:04.312893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:04.322021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:16:04.323622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:16:04.331945 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1186/2346] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:16:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:16:14.204679 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:14.328384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=14 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:16:49.594310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:16:49.595135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:16:49.595135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:16:49.614239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] MFA Rules not processed for user `1a98ea9b90174a24b57b3875e9df2ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:16:49.615541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:16:49.681843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1161/2347] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:16:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:16:59.672472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=25 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:01.436554 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:01.438092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:01.497880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470331a9-445f-49db-84ed-12b0c6d29b04 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:01.501685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:01.503479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:01.503479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:01.503479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:01.505205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:01.567917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:01.569555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:01.579144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1187/2348] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:11.574690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=50 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:24.240021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:24.241546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:24.253944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825d90-e711-49bd-889c-afe62fa83df2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:24.257570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63402a1-b4c8-4f67-a7fe-6c48269cf0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:24.258249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63402a1-b4c8-4f67-a7fe-6c48269cf0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:24.258538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63402a1-b4c8-4f67-a7fe-6c48269cf0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:24.258816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63402a1-b4c8-4f67-a7fe-6c48269cf0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:24.260539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d63402a1-b4c8-4f67-a7fe-6c48269cf0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:24.332307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63402a1-b4c8-4f67-a7fe-6c48269cf0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:24.333781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63402a1-b4c8-4f67-a7fe-6c48269cf0f9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:24.344819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1162/2349] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:27.858689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:27.867701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab5ab0-2207-4c23-960c-29db661c45da service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:27.871452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:27.872110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:27.872399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:27.872679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:27.874575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:27.890923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:27.892392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:27.904871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1188/2350] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:34.289825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:34.290378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:34.290378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:34.307466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:17:34.308629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:34.316555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1163/2351] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:17:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:17:34.358374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:34.370558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd7d84-0cc1-4904-a044-0e12a46f1877 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:34.375808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03681861-5d04-4a27-9926-28b33d941da2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:34.376504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681861-5d04-4a27-9926-28b33d941da2 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:34.376792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681861-5d04-4a27-9926-28b33d941da2 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:34.377077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681861-5d04-4a27-9926-28b33d941da2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:34.385609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03681861-5d04-4a27-9926-28b33d941da2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:34.387261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03681861-5d04-4a27-9926-28b33d941da2 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:34.395834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1189/2352] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:17:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:17:38.819291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:38.821575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:38.839390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56c249-8214-4b8e-ad15-b230d8fff4c4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:38.846060 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:38.847240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:38.847528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:38.848055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:38.849903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:38.923205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:38.925331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:38.934690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1164/2353] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:44.392056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:44.694022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764e187-7ed3-43db-b1c6-eb28851ebf3c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:44.694389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764e187-7ed3-43db-b1c6-eb28851ebf3c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:44.694743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7764e187-7ed3-43db-b1c6-eb28851ebf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:44.718551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7764e187-7ed3-43db-b1c6-eb28851ebf3c None None] MFA Rules not processed for user `bfbcc9b45bc341228bcc0f2192040224`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:17:44.720085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7764e187-7ed3-43db-b1c6-eb28851ebf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:44.808987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1190/2354] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:17:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:17:45.054597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:45.066753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44702e94-5ef5-4225-a182-03f9a70e7365 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:45.082064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:45.082737 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:45.083018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:45.083293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:45.087153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:45.104298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:45.105966 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:45.145657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1165/2355] 10.222.0.88 () {60 vars in 1396 bytes} [Thu Jun 16 16:17:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:45.488017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764e187-7ed3-43db-b1c6-eb28851ebf3c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:45.502563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7764e187-7ed3-43db-b1c6-eb28851ebf3c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:45.506937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:45.507619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:45.507894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:45.508178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:45.510277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:45.523442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:45.525035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:45.535907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1191/2356] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:45.954112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:45.956396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:46.046321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09b6e2-5bb5-43d1-9602-812c0c3714b4 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:46.051836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dcc4a-42f3-44e1-a6b8-8cde0e8325fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:46.052780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dcc4a-42f3-44e1-a6b8-8cde0e8325fa service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:46.053185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dcc4a-42f3-44e1-a6b8-8cde0e8325fa service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:46.053570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dcc4a-42f3-44e1-a6b8-8cde0e8325fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:46.055544 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-628dcc4a-42f3-44e1-a6b8-8cde0e8325fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:46.064031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:17:46.069371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dcc4a-42f3-44e1-a6b8-8cde0e8325fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:46.071404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dcc4a-42f3-44e1-a6b8-8cde0e8325fa service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:46.089130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1166/2357] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:54.532621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:54.547673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060b705-2e64-4333-8a8e-620fd4680ea9 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:54.553726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fba866-f27b-475a-9076-c60fabb77323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:54.554417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba866-f27b-475a-9076-c60fabb77323 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:54.554706 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba866-f27b-475a-9076-c60fabb77323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:54.554995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba866-f27b-475a-9076-c60fabb77323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:54.556845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6fba866-f27b-475a-9076-c60fabb77323 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:54.572230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fba866-f27b-475a-9076-c60fabb77323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:54.579042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fba866-f27b-475a-9076-c60fabb77323 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:54.591687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1192/2358] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:55.025984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa926241-0fc8-46b5-86ac-0f17e2053c97 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.026524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa926241-0fc8-46b5-86ac-0f17e2053c97 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.028423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa926241-0fc8-46b5-86ac-0f17e2053c97 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.030937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1167/2359] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:17:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 16:17:55.038948 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ab76-4ab3-42dc-b182-435ef1b15843 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.039485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ab76-4ab3-42dc-b182-435ef1b15843 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.039923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ab76-4ab3-42dc-b182-435ef1b15843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.081280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d9f9ab76-4ab3-42dc-b182-435ef1b15843 None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:17:55.083181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d9f9ab76-4ab3-42dc-b182-435ef1b15843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:55.093111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1193/2360] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:17:55.106059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa926241-0fc8-46b5-86ac-0f17e2053c97 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:55.107912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aa926241-0fc8-46b5-86ac-0f17e2053c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:55.175858 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa926241-0fc8-46b5-86ac-0f17e2053c97 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:55.179769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171363f7-a80d-4823-a086-768ddeca4c6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:55.180423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171363f7-a80d-4823-a086-768ddeca4c6c service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.180709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171363f7-a80d-4823-a086-768ddeca4c6c service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.180990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171363f7-a80d-4823-a086-768ddeca4c6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.192479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171363f7-a80d-4823-a086-768ddeca4c6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:55.193934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171363f7-a80d-4823-a086-768ddeca4c6c service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:55.204627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1168/2361] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:55.276621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d989d-71a6-4905-9943-89a4ababf2fd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.276866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d989d-71a6-4905-9943-89a4ababf2fd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.277134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d989d-71a6-4905-9943-89a4ababf2fd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.278602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1194/2362] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:17:55.284116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280a75d-bbb9-4b1e-862b-55a35160d44d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.284404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280a75d-bbb9-4b1e-862b-55a35160d44d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.284682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280a75d-bbb9-4b1e-862b-55a35160d44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.302934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4280a75d-bbb9-4b1e-862b-55a35160d44d None None] MFA Rules not processed for user `33e61ed31e0943e8baf9a85412860245`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:17:55.304126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4280a75d-bbb9-4b1e-862b-55a35160d44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:55.310636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1169/2363] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:17:55.322916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d989d-71a6-4905-9943-89a4ababf2fd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:55.335707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d989d-71a6-4905-9943-89a4ababf2fd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:55.340552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7a9d-8409-432c-8ff4-07e04b0be980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:55.341390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7a9d-8409-432c-8ff4-07e04b0be980 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.342138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7a9d-8409-432c-8ff4-07e04b0be980 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.342654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7a9d-8409-432c-8ff4-07e04b0be980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.351955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7a9d-8409-432c-8ff4-07e04b0be980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:55.353400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7a9d-8409-432c-8ff4-07e04b0be980 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:55.361559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1195/2364] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:17:55.447111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705cfe0-40fe-40df-aba6-857173c1c2d0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.447401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705cfe0-40fe-40df-aba6-857173c1c2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.447682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705cfe0-40fe-40df-aba6-857173c1c2d0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.449199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1170/2365] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:17:55.455050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890881d-62cf-48ef-943c-798ada5b2f4e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.455340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890881d-62cf-48ef-943c-798ada5b2f4e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.456127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890881d-62cf-48ef-943c-798ada5b2f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.475900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a890881d-62cf-48ef-943c-798ada5b2f4e None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:17:55.477103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a890881d-62cf-48ef-943c-798ada5b2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:17:55.486447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1196/2366] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:17:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:17:55.497310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705cfe0-40fe-40df-aba6-857173c1c2d0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:17:55.508420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705cfe0-40fe-40df-aba6-857173c1c2d0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:17:55.512411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa192a8-6c75-4697-9776-091e0f4caa8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:17:55.513067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa192a8-6c75-4697-9776-091e0f4caa8c service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:17:55.513669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa192a8-6c75-4697-9776-091e0f4caa8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:17:55.513669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa192a8-6c75-4697-9776-091e0f4caa8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:17:55.525069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa192a8-6c75-4697-9776-091e0f4caa8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:17:55.526535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa192a8-6c75-4697-9776-091e0f4caa8c service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:17:55.538721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1171/2367] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:17:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:05.470151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:05.533034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=38 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:09.850270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890881d-62cf-48ef-943c-798ada5b2f4e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:09.862452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890881d-62cf-48ef-943c-798ada5b2f4e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:09.868094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:09.868775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:09.869063 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:09.869346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:09.871177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:09.884870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:09.886518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:09.897591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1197/2368] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:14.891185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ff543-9943-4544-acc7-b626fcf047d3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:14.891740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ff543-9943-4544-acc7-b626fcf047d3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:14.891740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ff543-9943-4544-acc7-b626fcf047d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:14.910357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-485ff543-9943-4544-acc7-b626fcf047d3 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:18:14.911564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-485ff543-9943-4544-acc7-b626fcf047d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:14.918975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1172/2369] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:18:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:18:14.931552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:14.943544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5896b-4763-4a9e-8c03-dd48c52d25bd service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:14.948704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:14.949479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:14.949812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:14.950093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:14.960306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:14.962618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:14.970882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1198/2370] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:18:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:18:24.667785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ff543-9943-4544-acc7-b626fcf047d3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:24.680804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ff543-9943-4544-acc7-b626fcf047d3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:24.686196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:24.686876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:24.687184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:24.687456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:24.691607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:24.710345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:24.712543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:24.741131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:24.742359 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1173/2371] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:24.742876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:24.846931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d840f-701c-40ed-b792-72a38ba169cd service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:24.852452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:24.853331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:24.853619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:24.853924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] PATH_INFO: `/v3/users/1a98ea9b90174a24b57b3875e9df2ecd` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:24.855125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a98ea9b90174a24b57b3875e9df2ecd)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:24.857071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:24.913320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a98ea9b90174a24b57b3875e9df2ecd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:24.914430 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a98ea9b90174a24b57b3875e9df2ecd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:24.915156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a98ea9b90174a24b57b3875e9df2ecd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:24.928912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a98ea9b90174a24b57b3875e9df2ecd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:24.938578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a98ea9b90174a24b57b3875e9df2ecd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:24.958523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1199/2372] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:18:24 2022] DELETE /identity/v3/users/1a98ea9b90174a24b57b3875e9df2ecd => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:18:25.220696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:25.238073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8ecd7-c2bc-467f-8119-8f0c7741a2e6 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:25.319573 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:25.320422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:25.320695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:25.320967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] PATH_INFO: `/v3/projects/f5e363bcf5ed47529b6464cf954a31ea` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:25.322220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e363bcf5ed47529b6464cf954a31ea)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:25.324170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:25.381897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e363bcf5ed47529b6464cf954a31ea', 'request_id': 'req-d904f9c8-8bfe-4912-86aa-15fe81aaef65'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:25.397995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e363bcf5ed47529b6464cf954a31ea', 'request_id': 'req-d904f9c8-8bfe-4912-86aa-15fe81aaef65'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:25.407649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e363bcf5ed47529b6464cf954a31ea', 'request_id': 'req-d904f9c8-8bfe-4912-86aa-15fe81aaef65'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:25.422833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] The token cache is being invalidate because project f5e363bcf5ed47529b6464cf954a31ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:18:25.423202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:25.424042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d904f9c8-8bfe-4912-86aa-15fe81aaef65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:18:25.425672 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1174/2373] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:18:25 2022] DELETE /identity/v3/projects/f5e363bcf5ed47529b6464cf954a31ea => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:18:34.954065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=65,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:35.418045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:44.473857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:44.475383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:44.554399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85ce17-d523-4023-a3d0-216842227bd7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:44.619435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:44.620029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:44.620311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:44.620586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:44.622772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:44.634528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:44.636138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:44.646491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1200/2374] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:18:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:18:48.216090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:48.216559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:48.216660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:48.234786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:18:48.236474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:48.289579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1175/2375] 10.222.0.88 () {58 vars in 967 bytes} [Thu Jun 16 16:18:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:18:48.311073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:18:48.312465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:18:48.381538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca998e-a334-4bb8-b92f-6607938c8cc7 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:18:48.387230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:18:48.387889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:18:48.388176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:18:48.388451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:18:48.397761 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:18:48.399283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:18:48.407559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1201/2376] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:18:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:18:58.283283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=25 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:18:58.404153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:12.998918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:13.001036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:13.018262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5144f31-019f-4cf2-b662-9489087df84d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:13.024454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f7d9d-f1c7-4acf-8733-2ea7ed1bf2f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:13.025107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f7d9d-f1c7-4acf-8733-2ea7ed1bf2f2 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:13.025390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f7d9d-f1c7-4acf-8733-2ea7ed1bf2f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:13.025670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f7d9d-f1c7-4acf-8733-2ea7ed1bf2f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:13.027417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-240f7d9d-f1c7-4acf-8733-2ea7ed1bf2f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:13.086286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f7d9d-f1c7-4acf-8733-2ea7ed1bf2f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:13.087862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f7d9d-f1c7-4acf-8733-2ea7ed1bf2f2 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:13.096938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1176/2377] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:19:23.093204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:25.244788 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:25.256870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1836-b868-47cb-9a26-747c79fa7223 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:25.262723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b1954-5964-43f8-92ba-480b9d82c8b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:25.263372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b1954-5964-43f8-92ba-480b9d82c8b0 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:25.263659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b1954-5964-43f8-92ba-480b9d82c8b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:25.263938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b1954-5964-43f8-92ba-480b9d82c8b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:25.265767 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e6b1954-5964-43f8-92ba-480b9d82c8b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:25.331430 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b1954-5964-43f8-92ba-480b9d82c8b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:25.333051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b1954-5964-43f8-92ba-480b9d82c8b0 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:25.342338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1202/2378] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:19:35.338140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:42.428623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6ae7-032d-4740-963e-23114edcad23 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:42.429213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6ae7-032d-4740-963e-23114edcad23 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:42.429213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d6ae7-032d-4740-963e-23114edcad23 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:42.430605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1177/2379] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:19:42.436958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217e89-b2fe-4771-a8b9-8f8ee8fa0437 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:42.437461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217e89-b2fe-4771-a8b9-8f8ee8fa0437 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:42.437784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db217e89-b2fe-4771-a8b9-8f8ee8fa0437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:42.456528 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-db217e89-b2fe-4771-a8b9-8f8ee8fa0437 None None] MFA Rules not processed for user `33e61ed31e0943e8baf9a85412860245`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:42.457128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db217e89-b2fe-4771-a8b9-8f8ee8fa0437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:42.523126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1203/2380] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:19:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:19:42.542364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6ae7-032d-4740-963e-23114edcad23 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:42.544915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3c6d6ae7-032d-4740-963e-23114edcad23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:42.560126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d6ae7-032d-4740-963e-23114edcad23 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:42.573287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765e24e-a028-4c2f-b356-b19a5854ea1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:42.574014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765e24e-a028-4c2f-b356-b19a5854ea1e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:42.574394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765e24e-a028-4c2f-b356-b19a5854ea1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:42.574564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765e24e-a028-4c2f-b356-b19a5854ea1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:42.583524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765e24e-a028-4c2f-b356-b19a5854ea1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:42.584596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765e24e-a028-4c2f-b356-b19a5854ea1e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:42.601920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1178/2381] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:19:42.689499 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df340fa-671a-42df-9689-8cfcfc2708f7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:42.689823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df340fa-671a-42df-9689-8cfcfc2708f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:42.690188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df340fa-671a-42df-9689-8cfcfc2708f7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:42.691546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1204/2382] 10.222.0.88 () {54 vars in 921 bytes} [Thu Jun 16 16:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 16:19:42.698075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009767f-3c3b-4e1b-a69d-fa10e09b26d4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:42.698368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009767f-3c3b-4e1b-a69d-fa10e09b26d4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:42.698653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009767f-3c3b-4e1b-a69d-fa10e09b26d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:42.718159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2009767f-3c3b-4e1b-a69d-fa10e09b26d4 None None] MFA Rules not processed for user `ee056b132bd24851bd0e2afe9cb7170e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:42.719354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2009767f-3c3b-4e1b-a69d-fa10e09b26d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:42.729220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1179/2383] 10.222.0.88 () {58 vars in 1018 bytes} [Thu Jun 16 16:19:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:19:42.740069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df340fa-671a-42df-9689-8cfcfc2708f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:19:42.751533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df340fa-671a-42df-9689-8cfcfc2708f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:19:42.755875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a54679-c2ec-4fa4-a144-ef35ac384c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:19:42.756522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a54679-c2ec-4fa4-a144-ef35ac384c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:42.756808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a54679-c2ec-4fa4-a144-ef35ac384c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:42.757086 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a54679-c2ec-4fa4-a144-ef35ac384c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:42.768103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a54679-c2ec-4fa4-a144-ef35ac384c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:19:42.769550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a54679-c2ec-4fa4-a144-ef35ac384c39 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:19:42.780800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1205/2384] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:19:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:19:44.653961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61eac6-cab6-4498-a706-47d222de9061 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:19:44.654530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61eac6-cab6-4498-a706-47d222de9061 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:19:44.654622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61eac6-cab6-4498-a706-47d222de9061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:19:44.672180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2e61eac6-cab6-4498-a706-47d222de9061 None None] MFA Rules not processed for user `16b8389777744e6581ad14be85731f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:19:44.673391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2e61eac6-cab6-4498-a706-47d222de9061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:19:44.681212 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1180/2385] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:19:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:19:52.776774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=29 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:19:54.666809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=12 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:10.411526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:10.412201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:10.412201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:10.431850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] MFA Rules not processed for user `bfbcc9b45bc341228bcc0f2192040224`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:10.433100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:10.488301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1206/2386] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:20:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:20.481084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=25 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:21.687728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafceba-9bf9-464b-8918-5500592e6f86 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:21.688262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafceba-9bf9-464b-8918-5500592e6f86 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:21.688380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafceba-9bf9-464b-8918-5500592e6f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:21.707112 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-eaafceba-9bf9-464b-8918-5500592e6f86 None None] MFA Rules not processed for user `46445560ea7e496fa6c9c2e7c9a5b747`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:20:21.708309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eaafceba-9bf9-464b-8918-5500592e6f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:21.762247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1181/2387] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:20:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:20:22.129988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:22.131569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:22.144578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60509d61-5a04-4de9-a470-34f0986b083a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:22.148268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:22.148947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:22.149230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:22.149504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:22.151297 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:22.161061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:22.162522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:22.170573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1207/2388] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:28.205314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafceba-9bf9-464b-8918-5500592e6f86 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:20:28.219231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafceba-9bf9-464b-8918-5500592e6f86 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:20:28.225248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:20:28.225937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:20:28.226301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:20:28.226605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:20:28.230280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:20:28.242870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:20:28.244465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:20:28.257420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1182/2389] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:20:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:20:32.167074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:20:38.253581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=31 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:02.804629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:02.806168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:02.822246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876be548-0fee-410e-abdf-b5da144dd035 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:02.827839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c2967-b880-43eb-833e-a04835c0a95a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:02.828464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c2967-b880-43eb-833e-a04835c0a95a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:02.828749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c2967-b880-43eb-833e-a04835c0a95a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:02.829026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c2967-b880-43eb-833e-a04835c0a95a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:02.830828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d46c2967-b880-43eb-833e-a04835c0a95a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:02.889060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c2967-b880-43eb-833e-a04835c0a95a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:02.890716 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c2967-b880-43eb-833e-a04835c0a95a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:02.899571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1208/2390] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:12.877936 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:12.879430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:12.895453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:12.951652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c065e-1bd0-4e2d-b193-c29c13eccfce service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:12.956734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:12.957549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:12.957687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:12.957995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:12.960055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:12.973088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:12.974722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:12.985225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1183/2391] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:13.769226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d328c6-8d2a-4f80-b8f0-c4c14e1035bc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:13.769520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d328c6-8d2a-4f80-b8f0-c4c14e1035bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:13.769856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d328c6-8d2a-4f80-b8f0-c4c14e1035bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:13.786827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f2d328c6-8d2a-4f80-b8f0-c4c14e1035bc None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:13.788032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f2d328c6-8d2a-4f80-b8f0-c4c14e1035bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:13.795120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1209/2392] 10.222.0.88 () {58 vars in 967 bytes} [Thu Jun 16 16:21:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:13.842453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:13.854153 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3443b-0c99-4dc5-8b93-f1bc0e988b09 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:13.859187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:13.859824 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:13.860111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:13.860393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:13.868600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:13.870746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:13.878855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1184/2393] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:21:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:21:14.214780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d328c6-8d2a-4f80-b8f0-c4c14e1035bc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:14.224413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d328c6-8d2a-4f80-b8f0-c4c14e1035bc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:14.227988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:14.228627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:14.228910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:14.229709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:14.231491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:14.295490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:14.296880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:14.306422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1210/2394] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:20.569416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:20.572378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:20.593049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4664-9f00-4478-8a97-59dedc3771d6 service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:20.607795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:20.608634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:20.609052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:20.609570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:20.612619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:20.627181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:20.628668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:20.636982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:20.638500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:20.657549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672f90b-85f3-482d-b395-6bf6caf396a8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:20.661100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1185/2395] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:20.664253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:20.665388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:20.665684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:20.665999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] PATH_INFO: `/v3/users/6e097835e6604a02a38b291786033f06` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:20.667724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e097835e6604a02a38b291786033f06)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:20.669669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:20.736076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e097835e6604a02a38b291786033f06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:20.736794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e097835e6604a02a38b291786033f06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:20.737504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e097835e6604a02a38b291786033f06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:20.746820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e097835e6604a02a38b291786033f06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:20.753804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e097835e6604a02a38b291786033f06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:20.773798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1211/2396] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:21:20 2022] DELETE /identity/v3/users/6e097835e6604a02a38b291786033f06 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:20.956008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:20.967087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a3981-31d8-4d17-a1d7-8cc2bed8705e service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:21.024558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:21.025416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:21.025701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:21.026564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] PATH_INFO: `/v3/projects/731df982fd964de6bc4545103f7e1047` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:21.027768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=731df982fd964de6bc4545103f7e1047)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:21.029683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:21.076570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '731df982fd964de6bc4545103f7e1047', 'request_id': 'req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.083674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '731df982fd964de6bc4545103f7e1047', 'request_id': 'req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.089937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '731df982fd964de6bc4545103f7e1047', 'request_id': 'req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.093771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:21.102049 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] The token cache is being invalidate because project 731df982fd964de6bc4545103f7e1047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:21:21.102389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.103336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.105596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1186/2397] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:21:20 2022] DELETE /identity/v3/projects/731df982fd964de6bc4545103f7e1047 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:21.109668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:21.111088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:21.123933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef308b-ecd2-4a64-9c2a-625087cd6f89 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:21.181950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:21.182847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:21.183114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:21.183394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] PATH_INFO: `/v3/users/57015b7bef2a478687ca669af612fcaf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:21.184575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=57015b7bef2a478687ca669af612fcaf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:21.186544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:21.187980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:21.235366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57015b7bef2a478687ca669af612fcaf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.236050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57015b7bef2a478687ca669af612fcaf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.236646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57015b7bef2a478687ca669af612fcaf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.244019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57015b7bef2a478687ca669af612fcaf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.252277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57015b7bef2a478687ca669af612fcaf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.270355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1212/2398] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:21:21 2022] DELETE /identity/v3/users/57015b7bef2a478687ca669af612fcaf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:21.445937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:21.447482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:21.467413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b793f-3d3b-4f52-9afd-a7ceee390e15 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:21.524750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:21.525603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:21.525917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:21.526196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] PATH_INFO: `/v3/projects/4ee4194f9bfb4acb93f9b0c48e270202` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:21.527366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee4194f9bfb4acb93f9b0c48e270202)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:21.529301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:21.576639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee4194f9bfb4acb93f9b0c48e270202', 'request_id': 'req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.583415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee4194f9bfb4acb93f9b0c48e270202', 'request_id': 'req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.590371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee4194f9bfb4acb93f9b0c48e270202', 'request_id': 'req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.601440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] The token cache is being invalidate because project 4ee4194f9bfb4acb93f9b0c48e270202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:21:21.601752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.602401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:21.604729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1187/2399] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:21:21 2022] DELETE /identity/v3/projects/4ee4194f9bfb4acb93f9b0c48e270202 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:31.273950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:31.598095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:35.819117 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:35.820633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:35.877642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a483a-3e71-41ed-91cd-e7dd469acf2e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:35.924539 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:35.925184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:35.925471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:35.925777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:35.927556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:35.987577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:35.989014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:35.999140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1213/2400] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:39.490003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:39.491680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:39.505591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2ab22-2ea1-4e15-adde-ae3727d01f78 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:39.512229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:39.513144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:39.513437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:39.513815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] PATH_INFO: `/v3/users/16b8389777744e6581ad14be85731f4a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:39.515022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b8389777744e6581ad14be85731f4a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:39.516983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:39.565957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b8389777744e6581ad14be85731f4a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.566729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b8389777744e6581ad14be85731f4a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.574007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b8389777744e6581ad14be85731f4a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.574583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b8389777744e6581ad14be85731f4a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.581952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6760f966-14c1-49cd-bf03-4b8aed8c947f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b8389777744e6581ad14be85731f4a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.600340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1188/2401] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:21:39 2022] DELETE /identity/v3/users/16b8389777744e6581ad14be85731f4a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:39.778264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:39.789094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba2c6e-166b-4ea7-95c6-0924bc6c1bb8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:39.845375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:39.846215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:39.846506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:39.846790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] PATH_INFO: `/v3/projects/c3592ee016d1486db983b4ff2aab5ca3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:39.848220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3592ee016d1486db983b4ff2aab5ca3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:39.850178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:39.858621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:39.897747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3592ee016d1486db983b4ff2aab5ca3', 'request_id': 'req-b5799fa1-af5c-4179-8b4f-447ed71be47e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.905489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3592ee016d1486db983b4ff2aab5ca3', 'request_id': 'req-b5799fa1-af5c-4179-8b4f-447ed71be47e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.911857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3592ee016d1486db983b4ff2aab5ca3', 'request_id': 'req-b5799fa1-af5c-4179-8b4f-447ed71be47e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.922514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] The token cache is being invalidate because project c3592ee016d1486db983b4ff2aab5ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:21:39.922804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.923442 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5799fa1-af5c-4179-8b4f-447ed71be47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:21:39.924980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1214/2402] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:21:39 2022] DELETE /identity/v3/projects/c3592ee016d1486db983b4ff2aab5ca3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:39.930013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930365b-ccfc-4af9-8e1a-5fafcdc3cecf None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:39.930307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930365b-ccfc-4af9-8e1a-5fafcdc3cecf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:39.930589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930365b-ccfc-4af9-8e1a-5fafcdc3cecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:39.948378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4930365b-ccfc-4af9-8e1a-5fafcdc3cecf None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:39.949568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4930365b-ccfc-4af9-8e1a-5fafcdc3cecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:40.008106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1189/2403] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:40.016161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07774-cb7c-41fa-b15c-819c3564f662 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.016487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07774-cb7c-41fa-b15c-819c3564f662 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.016779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d07774-cb7c-41fa-b15c-819c3564f662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.036534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-53d07774-cb7c-41fa-b15c-819c3564f662 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:40.037737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53d07774-cb7c-41fa-b15c-819c3564f662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:40.046150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1215/2404] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:40.050356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930365b-ccfc-4af9-8e1a-5fafcdc3cecf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.060862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930365b-ccfc-4af9-8e1a-5fafcdc3cecf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.065550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.066297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.066615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.066897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.067626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:21:40.067938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.069579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.081356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1190/2405] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:21:40.085707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07774-cb7c-41fa-b15c-819c3564f662 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.095261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d07774-cb7c-41fa-b15c-819c3564f662 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.100215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.100802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.101088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.101367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.102087 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.104547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.139348 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1216/2406] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:21:40 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:21:40.143452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.153856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058777-f23d-4284-b4a8-8b27a1eb132c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.209543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.210231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.210518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.210798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.211494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.213395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.219853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:40.234257 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1191/2407] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:21:40 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:21:40.238303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.248904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440e3f2-043e-4afd-b2ac-d466375404f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.253485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.254166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.254454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.254837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.255463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.257075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.267597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1217/2408] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:21:40.271230 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.281201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbf078-4a25-4e33-bd2d-e0d629ebfb72 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.285835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bcc6c-037c-44b1-988f-e912a62a1ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.286498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bcc6c-037c-44b1-988f-e912a62a1ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.286785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bcc6c-037c-44b1-988f-e912a62a1ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.287065 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bcc6c-037c-44b1-988f-e912a62a1ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.287818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bcc6c-037c-44b1-988f-e912a62a1ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.289429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bcc6c-037c-44b1-988f-e912a62a1ea4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.299878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1192/2409] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:21:40.304019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.314740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d994fb-3250-4120-bfad-68be6bed6ad1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.319347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.320268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.320550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.320825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] PATH_INFO: `/v3/projects/41f6c4a1b7784d7d874b00d5e43507cf/users/c6c53bf1554a4825afd8770ea9347ad1/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.335823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f6c4a1b7784d7d874b00d5e43507cf, user_id=c6c53bf1554a4825afd8770ea9347ad1, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.338523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.351576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1218/2410] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:21:40 2022] PUT /identity/v3/projects/41f6c4a1b7784d7d874b00d5e43507cf/users/c6c53bf1554a4825afd8770ea9347ad1/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:21:40.356983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5efad-1c93-4d50-9e03-ac8350ab122b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.357280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5efad-1c93-4d50-9e03-ac8350ab122b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.357566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5efad-1c93-4d50-9e03-ac8350ab122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.375676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-07e5efad-1c93-4d50-9e03-ac8350ab122b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:40.376879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-07e5efad-1c93-4d50-9e03-ac8350ab122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:40.436768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1193/2411] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:40.447197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:40.448923 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:40.507295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e70e5-fa9b-4523-b0c2-48b1b7747819 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:40.511111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:40.511766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:40.512055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:40.512337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:40.521950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:40.523405 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:40.533031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1219/2412] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:44.178067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63aba-b5cd-4544-9038-0c2a0bf2ca89 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:44.178610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63aba-b5cd-4544-9038-0c2a0bf2ca89 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:44.178610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c63aba-b5cd-4544-9038-0c2a0bf2ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:44.207149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c0c63aba-b5cd-4544-9038-0c2a0bf2ca89 None None] MFA Rules not processed for user `c6c53bf1554a4825afd8770ea9347ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:44.208255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c0c63aba-b5cd-4544-9038-0c2a0bf2ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:44.274666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1194/2413] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:21:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:44.289121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:44.291119 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:44.322825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:44.367584 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6992a7-a765-4954-bb95-cdc2eefccdc5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:44.373421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:44.374108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:44.374399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:44.374679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:44.382580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:44.384715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:44.394370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1220/2414] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:21:45.756498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:45.757026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:45.757142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:45.776531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] MFA Rules not processed for user `c6c53bf1554a4825afd8770ea9347ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:21:45.778346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:45.791254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1195/2415] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:21:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:21:54.390279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=22 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:55.770450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:21:59.495930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:21:59.497562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:59.510887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424f76e-ff47-4481-8088-0d417ebab4af service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:21:59.514710 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:21:59.515387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:21:59.515677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:21:59.515955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:21:59.517730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:21:59.530826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:21:59.532207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:21:59.543212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1221/2416] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:22:05.971790 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:05.973778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:05.988133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e542a-702b-49ee-99df-ebe18ab4b1ff None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:05.992701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:05.993603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:05.993935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:05.994217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] PATH_INFO: `/v3/users/578e9a80847d4fb18e9ea43a3ed7e8f5` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:05.995436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=578e9a80847d4fb18e9ea43a3ed7e8f5)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:05.997403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:06.044234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '578e9a80847d4fb18e9ea43a3ed7e8f5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.044885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '578e9a80847d4fb18e9ea43a3ed7e8f5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.051952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '578e9a80847d4fb18e9ea43a3ed7e8f5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.052601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '578e9a80847d4fb18e9ea43a3ed7e8f5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.060369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '578e9a80847d4fb18e9ea43a3ed7e8f5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.080421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1196/2417] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:22:05 2022] DELETE /identity/v3/users/578e9a80847d4fb18e9ea43a3ed7e8f5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:22:06.257084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:06.267242 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a420e27-3999-4162-8264-18014bc169e8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:06.324678 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:06.325568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:06.325887 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:06.326171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] PATH_INFO: `/v3/projects/aed429e2356c4c7a83872d82c0540b66` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:06.327338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed429e2356c4c7a83872d82c0540b66)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:06.329283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:06.374683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed429e2356c4c7a83872d82c0540b66', 'request_id': 'req-9c130f49-fa5e-41d5-9525-9c969978fd0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.382007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed429e2356c4c7a83872d82c0540b66', 'request_id': 'req-9c130f49-fa5e-41d5-9525-9c969978fd0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.389221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed429e2356c4c7a83872d82c0540b66', 'request_id': 'req-9c130f49-fa5e-41d5-9525-9c969978fd0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.399509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] The token cache is being invalidate because project aed429e2356c4c7a83872d82c0540b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:22:06.399798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.400439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.402017 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1222/2418] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:22:06 2022] DELETE /identity/v3/projects/aed429e2356c4c7a83872d82c0540b66 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:22:06.406249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:06.407700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:06.420736 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac05438-bb46-4c5d-97cb-5cdefdcacde9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:06.477731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:06.478629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:06.478919 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:06.479202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] PATH_INFO: `/v3/users/ecd4e8fc68384c6c9c0b4f22f3c9c84d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:06.480393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd4e8fc68384c6c9c0b4f22f3c9c84d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:06.482365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:06.528910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd4e8fc68384c6c9c0b4f22f3c9c84d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.529577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd4e8fc68384c6c9c0b4f22f3c9c84d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.536338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd4e8fc68384c6c9c0b4f22f3c9c84d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.536971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd4e8fc68384c6c9c0b4f22f3c9c84d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.544267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd4e8fc68384c6c9c0b4f22f3c9c84d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.552050 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:06.563477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1197/2419] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:22:06 2022] DELETE /identity/v3/users/ecd4e8fc68384c6c9c0b4f22f3c9c84d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:22:06.739368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:06.740799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:06.754142 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c130f49-fa5e-41d5-9525-9c969978fd0d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:06.811489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:06.812271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:06.812555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:06.812863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] PATH_INFO: `/v3/projects/628ab5a8f33e40048cfc723209f0280e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:06.814049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=628ab5a8f33e40048cfc723209f0280e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:06.815971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:06.837116 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:06.865484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628ab5a8f33e40048cfc723209f0280e', 'request_id': 'req-f3bd20d7-2808-4dcf-9be1-03d2019217ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.873645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628ab5a8f33e40048cfc723209f0280e', 'request_id': 'req-f3bd20d7-2808-4dcf-9be1-03d2019217ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.881999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628ab5a8f33e40048cfc723209f0280e', 'request_id': 'req-f3bd20d7-2808-4dcf-9be1-03d2019217ba'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.892565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] The token cache is being invalidate because project 628ab5a8f33e40048cfc723209f0280e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:22:06.893252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.893661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f3bd20d7-2808-4dcf-9be1-03d2019217ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:22:06.895397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1223/2420] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:22:06 2022] DELETE /identity/v3/projects/628ab5a8f33e40048cfc723209f0280e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:22:16.561621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:16.889169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:23.242436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:23.243877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:23.304469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650fc89-7d54-4ed9-a322-5ea17f074412 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:23.353007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:23.353644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:23.353955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:23.354229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:23.355989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:23.435224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:23.436696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:23.451192 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1198/2421] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:22:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:22:28.812325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:28.812802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:28.812914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:28.855326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] MFA Rules not processed for user `bfbcc9b45bc341228bcc0f2192040224`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:22:28.856529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:28.918984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1224/2422] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:22:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:22:28.954852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:28.959366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:29.016724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4532908-4a8f-4938-a280-3b8f9ea4521d service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:29.020566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:29.021299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:29.021585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:29.021899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:29.023807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:29.037277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:29.039290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:29.044879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:29.052005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1199/2423] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:22:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:22:37.728544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:37.730024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:37.745925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e44e60-3d22-493f-a81c-cb9933d1b4e9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:37.751296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd6f66-9686-4922-a4e5-6cb3cb012a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:37.751930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6f66-9686-4922-a4e5-6cb3cb012a41 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:37.752216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6f66-9686-4922-a4e5-6cb3cb012a41 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:37.752493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd6f66-9686-4922-a4e5-6cb3cb012a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:37.754227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-06cd6f66-9686-4922-a4e5-6cb3cb012a41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:37.836175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6f66-9686-4922-a4e5-6cb3cb012a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:37.837912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd6f66-9686-4922-a4e5-6cb3cb012a41 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:37.847850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1225/2424] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:22:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:22:44.336498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:44.338044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:44.358515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd615-6e2d-408b-bb91-1b23a6ae278d service glance] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:44.364234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:44.364814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:44.365233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:44.365525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:44.367820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:44.383091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:44.385146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:44.394593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1200/2425] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:22:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:22:44.685026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:44.685527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:44.685646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:44.703570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:22:44.704814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:44.762725 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1226/2426] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:22:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:22:44.810341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:44.811884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:44.898913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e782e-5078-40ad-a2d8-6cc5ae7ee2e7 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:44.914915 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:44.915585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:44.915870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:44.916177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:44.925564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:44.927188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:44.936064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1201/2427] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:22:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:22:54.755168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:54.932772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=38 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:22:56.091420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:22:56.092859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:56.104657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e6418-1ebe-4989-ac77-b4610cfdd784 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:22:56.108243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:22:56.108884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:22:56.109169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:22:56.109447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:22:56.111194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:22:56.120399 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:22:56.121878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:22:56.130317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1227/2428] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:22:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:23:00.728467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:00.738396 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376acef-15ce-4821-9b70-c673c83a5d76 service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:00.742772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:00.743415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:00.743697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:00.743997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:00.745803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:00.760677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:00.762079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:00.775637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1202/2429] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:23:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:23:00.842701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:00.844164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:00.854675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ffb7b-0066-4cb8-9926-44ebeed86255 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:00.859164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:00.860176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:00.860525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:00.860869 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:00.862992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:00.874178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:00.875637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:00.883613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1228/2430] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:23:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:23:00.954750 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:00.964606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2682b2-b527-41f4-8668-45755bbdb781 service glance] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:00.968385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472175d1-7277-4971-b878-e4b4cf540648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:00.969021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472175d1-7277-4971-b878-e4b4cf540648 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:00.969308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472175d1-7277-4971-b878-e4b4cf540648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:00.969588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472175d1-7277-4971-b878-e4b4cf540648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:00.971369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-472175d1-7277-4971-b878-e4b4cf540648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:00.985340 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472175d1-7277-4971-b878-e4b4cf540648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:00.986811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472175d1-7277-4971-b878-e4b4cf540648 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:00.997574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1203/2431] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:23:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:23:01.057585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:01.059049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:01.128027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68676ad-f1b5-432b-be6f-639a8509f1ed service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:01.133657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:01.134336 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:01.134625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:01.134901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:01.136839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:01.149627 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:01.151275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:01.161133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1229/2432] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:23:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:23:02.752207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2075-57e2-4c2f-8fbc-f621cb45c936 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:02.752675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2075-57e2-4c2f-8fbc-f621cb45c936 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:02.752777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2075-57e2-4c2f-8fbc-f621cb45c936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:02.770481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d78a2075-57e2-4c2f-8fbc-f621cb45c936 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:23:02.771719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d78a2075-57e2-4c2f-8fbc-f621cb45c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:02.779418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1204/2433] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:23:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:23:02.791255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:02.802569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a6335-1e47-4f65-bd1d-51bfdd309b03 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:02.807462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:02.808096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:02.808384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:02.808665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:02.817126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:02.819284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:02.827554 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1230/2434] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:23:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 16:23:12.765553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=15 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:23:12.824234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=50 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:23:24.482920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0e80-91e6-4eb9-a183-5caba1c41dc4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:24.483381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0e80-91e6-4eb9-a183-5caba1c41dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:24.483524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0e80-91e6-4eb9-a183-5caba1c41dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:24.502040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9abb0e80-91e6-4eb9-a183-5caba1c41dc4 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:23:24.503418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9abb0e80-91e6-4eb9-a183-5caba1c41dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:23:24.510593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1205/2435] 10.222.0.88 () {58 vars in 967 bytes} [Thu Jun 16 16:23:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:23:24.522721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:23:24.533387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a46c10-d70e-48e9-a8a5-5d49de708a23 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:23:24.539232 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:23:24.540146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:23:24.540435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:23:24.540713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:23:24.549042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:23:24.550697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:23:24.560173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1231/2436] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:23:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:23:34.497262 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:23:34.556408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:29.493543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0e80-91e6-4eb9-a183-5caba1c41dc4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:29.508554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0e80-91e6-4eb9-a183-5caba1c41dc4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:29.514621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:29.515410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:29.515719 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:29.515981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:29.518093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:29.531808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:29.533418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:29.545161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1206/2437] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:35.019834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:35.030368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e46e8-e53b-4856-8446-6b3d73106c79 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:35.035846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:35.036534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:35.036821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:35.037104 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:35.038942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:35.119187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:35.120640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:35.130961 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1232/2438] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:38.630338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:38.632316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:38.646550 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16af984-6fb2-4312-bc63-c2ec1f48d097 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:38.651190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:38.652082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:38.652388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:38.652682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] PATH_INFO: `/v3/users/bfbcc9b45bc341228bcc0f2192040224` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:38.653930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfbcc9b45bc341228bcc0f2192040224)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:38.655882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:38.705741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfbcc9b45bc341228bcc0f2192040224'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:38.706333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfbcc9b45bc341228bcc0f2192040224'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:38.713066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfbcc9b45bc341228bcc0f2192040224'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:38.713687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfbcc9b45bc341228bcc0f2192040224'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:38.721538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c77f0ca-f40e-44a3-9f50-9f8d4854fbd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfbcc9b45bc341228bcc0f2192040224'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:38.741407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1207/2439] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:24:38 2022] DELETE /identity/v3/users/bfbcc9b45bc341228bcc0f2192040224 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:38.922398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:38.932933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac62bec-716a-4d63-b154-403f33e01a8d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:38.998663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:38.999567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:38.999850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.000151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] PATH_INFO: `/v3/projects/c1cdf363c2e44af5b0d8cd748c58235b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.001333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1cdf363c2e44af5b0d8cd748c58235b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.003925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.054637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1cdf363c2e44af5b0d8cd748c58235b', 'request_id': 'req-b7b7156e-367a-4959-acf7-1e7e848b7c2c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:39.062044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1cdf363c2e44af5b0d8cd748c58235b', 'request_id': 'req-b7b7156e-367a-4959-acf7-1e7e848b7c2c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:39.068673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1cdf363c2e44af5b0d8cd748c58235b', 'request_id': 'req-b7b7156e-367a-4959-acf7-1e7e848b7c2c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:39.079756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] The token cache is being invalidate because project c1cdf363c2e44af5b0d8cd748c58235b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:24:39.080041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:39.081248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7b7156e-367a-4959-acf7-1e7e848b7c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:39.082829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1233/2440] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:24:38 2022] DELETE /identity/v3/projects/c1cdf363c2e44af5b0d8cd748c58235b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:39.093096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861651-02f3-4ec6-98e1-69a1a5bbecda None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.093387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861651-02f3-4ec6-98e1-69a1a5bbecda None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.093669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69861651-02f3-4ec6-98e1-69a1a5bbecda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.111434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-69861651-02f3-4ec6-98e1-69a1a5bbecda None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:39.112634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69861651-02f3-4ec6-98e1-69a1a5bbecda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:39.171543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1208/2441] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:39.179270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ecb56-e07a-4c24-82b0-8da163005ac3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.179562 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ecb56-e07a-4c24-82b0-8da163005ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.179981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ecb56-e07a-4c24-82b0-8da163005ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.199124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2f1ecb56-e07a-4c24-82b0-8da163005ac3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:39.200721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2f1ecb56-e07a-4c24-82b0-8da163005ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:39.209091 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1234/2442] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:39.213326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69861651-02f3-4ec6-98e1-69a1a5bbecda None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.223761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69861651-02f3-4ec6-98e1-69a1a5bbecda None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.228455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.229116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.229404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.229703 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.230450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:24:39.230759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.232357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.244453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1209/2443] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:24:39.248459 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ecb56-e07a-4c24-82b0-8da163005ac3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.257604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ecb56-e07a-4c24-82b0-8da163005ac3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.262295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.263287 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.263577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.263855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.264550 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.266492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.280503 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:39.294566 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1235/2444] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:39 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:39.299003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.309326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefe0da-dd60-4328-bf85-b536237b5b25 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.360801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:39.369488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.370438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.370883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.371715 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.372646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.375207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.399067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1210/2445] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:39 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:39.403296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.413809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349efeb-6cd2-47ed-9c4f-bc94d9ef1af8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.423614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.423614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.423614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.423614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.426118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.426118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.430260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.431972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:39.438350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1236/2446] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:24:39.442055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.451655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd079e8d-bd8d-4a83-a3c9-4299e1a6ed50 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.456806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.457448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.457802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.458083 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.458793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.460383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.470433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1237/2447] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:24:39.474875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.484598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6de64-065a-4e15-ae12-315b7a1a6fd4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.489661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667bb9-4311-4f25-b191-15f0349cb4cb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.489860 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.490758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.491051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.491328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] PATH_INFO: `/v3/projects/12a1c871a00440e5ade5efea6883954a/users/2f8c4f1f7e9944c8ae1b490c38eeeaa5/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.493816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47553d5-cb3c-4b0c-8b0a-1d7809d6be32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.494462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47553d5-cb3c-4b0c-8b0a-1d7809d6be32 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.494798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47553d5-cb3c-4b0c-8b0a-1d7809d6be32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.495077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47553d5-cb3c-4b0c-8b0a-1d7809d6be32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.496827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d47553d5-cb3c-4b0c-8b0a-1d7809d6be32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:39.506371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a1c871a00440e5ade5efea6883954a, user_id=2f8c4f1f7e9944c8ae1b490c38eeeaa5, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.508971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47553d5-cb3c-4b0c-8b0a-1d7809d6be32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.509099 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.510432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47553d5-cb3c-4b0c-8b0a-1d7809d6be32 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.521367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1211/2448] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:39.522718 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1238/2449] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:39 2022] PUT /identity/v3/projects/12a1c871a00440e5ade5efea6883954a/users/2f8c4f1f7e9944c8ae1b490c38eeeaa5/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:39.532493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.532785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.533062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.559264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] MFA Rules not processed for user `2f8c4f1f7e9944c8ae1b490c38eeeaa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:39.560533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:39.625120 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1212/2450] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:39.635633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:39.637314 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:39.694255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073696d-29b1-4ca3-b202-e960c050742a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:39.697542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:39.698220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:39.698501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:39.698774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:39.707704 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:39.709139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:39.717981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1239/2451] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:42.579581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:42.582714 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:42.670001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:42.672804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:42.690081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b7ea0-c7d4-4f1d-bb7a-f696d4e2b815 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:42.696325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:42.696992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:42.697280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:42.697560 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:42.704515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:42.726018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:42.727989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:42.733363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:42.741218 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1213/2452] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:42.767742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d74cd-c72e-431c-9946-64368e31565c service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:42.772806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:42.773696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:42.774032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:42.774315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] PATH_INFO: `/v3/users/46445560ea7e496fa6c9c2e7c9a5b747` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:42.775877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46445560ea7e496fa6c9c2e7c9a5b747)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:42.777897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:42.833147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46445560ea7e496fa6c9c2e7c9a5b747'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:42.834250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46445560ea7e496fa6c9c2e7c9a5b747'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:42.834802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46445560ea7e496fa6c9c2e7c9a5b747'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:42.846451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46445560ea7e496fa6c9c2e7c9a5b747'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:42.848874 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:42.860285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ebb92ee6-0d3b-4a6d-a70d-c327ab5de993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46445560ea7e496fa6c9c2e7c9a5b747'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:42.886217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1240/2453] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:24:42 2022] DELETE /identity/v3/users/46445560ea7e496fa6c9c2e7c9a5b747 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:43.078128 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.090148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492a6c5-92da-4e1f-9da2-f04ab57dbd04 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.151538 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.152425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.152689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.152964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] PATH_INFO: `/v3/projects/dbfb390720a54a4eb35c80b0d1a1f16b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.154181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbfb390720a54a4eb35c80b0d1a1f16b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.156070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.201431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbfb390720a54a4eb35c80b0d1a1f16b', 'request_id': 'req-81bee8bf-858d-4411-acdc-433fe17d9d26'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:43.208554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbfb390720a54a4eb35c80b0d1a1f16b', 'request_id': 'req-81bee8bf-858d-4411-acdc-433fe17d9d26'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:43.214643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbfb390720a54a4eb35c80b0d1a1f16b', 'request_id': 'req-81bee8bf-858d-4411-acdc-433fe17d9d26'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:43.228551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] The token cache is being invalidate because project dbfb390720a54a4eb35c80b0d1a1f16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:24:43.228841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:43.229481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-81bee8bf-858d-4411-acdc-433fe17d9d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:43.231081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1214/2454] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:24:43 2022] DELETE /identity/v3/projects/dbfb390720a54a4eb35c80b0d1a1f16b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:43.235963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96134db-e08b-4e56-ad9b-37fd6774fb85 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.236741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96134db-e08b-4e56-ad9b-37fd6774fb85 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.237032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96134db-e08b-4e56-ad9b-37fd6774fb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.255081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b96134db-e08b-4e56-ad9b-37fd6774fb85 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:43.256282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b96134db-e08b-4e56-ad9b-37fd6774fb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:43.316100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1241/2455] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:43.324695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55820bf-f7a3-4b6d-9719-c87afbbd78dd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.324927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55820bf-f7a3-4b6d-9719-c87afbbd78dd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.325209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55820bf-f7a3-4b6d-9719-c87afbbd78dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.343061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e55820bf-f7a3-4b6d-9719-c87afbbd78dd None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:43.344244 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e55820bf-f7a3-4b6d-9719-c87afbbd78dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:43.352242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1215/2456] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:43.356434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96134db-e08b-4e56-ad9b-37fd6774fb85 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.366465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96134db-e08b-4e56-ad9b-37fd6774fb85 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.371025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.371675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.371960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.372235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.372953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:24:43.373264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.374894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.386662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1242/2457] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:24:43.390920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55820bf-f7a3-4b6d-9719-c87afbbd78dd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.400057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55820bf-f7a3-4b6d-9719-c87afbbd78dd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.404779 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.405421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.405700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.406005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.406693 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.408572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.433477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1216/2458] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:43 2022] POST /identity/v3/projects => generated 381 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:43.437583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.448274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e902ba8-2e68-4c1a-93ff-a63005e7a3db admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.508441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.509110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.509398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.509688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.510398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.512332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.532092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1243/2459] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:43 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:43.536687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.546795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d67c83-152d-460a-9a5d-f61ada5f10b7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.551900 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.552547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.552859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.553135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.553880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.555498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.566243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1217/2460] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:43.570113 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.580746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cb624-e3af-4de3-94bb-5bba63b4d6d3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.585185 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60f17e-09c8-4149-b9af-888cb82059cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.585871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60f17e-09c8-4149-b9af-888cb82059cd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.586153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60f17e-09c8-4149-b9af-888cb82059cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.586428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60f17e-09c8-4149-b9af-888cb82059cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.587140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60f17e-09c8-4149-b9af-888cb82059cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.588723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60f17e-09c8-4149-b9af-888cb82059cd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.598717 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1244/2461] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:43.602689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.612687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5564d-f043-4a61-9d41-4232d36ae677 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.617422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.618378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.618661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.618939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] PATH_INFO: `/v3/projects/c6db296a114e4480aa581d24d5192a41/users/f8e1943fd89a40dc9d843431e36512b1/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.634843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6db296a114e4480aa581d24d5192a41, user_id=f8e1943fd89a40dc9d843431e36512b1, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.638238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.651911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1218/2462] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:43 2022] PUT /identity/v3/projects/c6db296a114e4480aa581d24d5192a41/users/f8e1943fd89a40dc9d843431e36512b1/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:43.662121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457abe6e-c8ac-40ff-b435-211cc72f7dd6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.662416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457abe6e-c8ac-40ff-b435-211cc72f7dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.662717 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457abe6e-c8ac-40ff-b435-211cc72f7dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.688066 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-457abe6e-c8ac-40ff-b435-211cc72f7dd6 None None] MFA Rules not processed for user `f8e1943fd89a40dc9d843431e36512b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:43.689244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-457abe6e-c8ac-40ff-b435-211cc72f7dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:43.715700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:43.757613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1245/2463] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:43.762125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.790692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:43.828512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0abdf-f939-41f2-818e-599e1037ee23 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.833842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.834522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.834811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.835094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.835794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.837763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.862903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1219/2464] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:43 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:43.867166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457abe6e-c8ac-40ff-b435-211cc72f7dd6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.878057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457abe6e-c8ac-40ff-b435-211cc72f7dd6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.935121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.936005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.936331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.936568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.937259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.939275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:43.961616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1246/2465] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:43 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:43.965606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:43.976807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fc0e0-fff7-468f-abd1-cba7dc5e3d8f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:43.981993 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:43.982749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:43.983131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:43.983505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:43.984318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:43.986014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:44.000429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1220/2466] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:44.004246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:44.014481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43950465-2648-42da-89de-56ba40dcaf4d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:44.019818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:44.020740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:44.021005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:44.021404 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] PATH_INFO: `/v3/projects/75d0ace1a04c4a65903494f7e6d96cca/users/cf71a4aec63e4229a8bf9840dd5c5c51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:44.037466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d0ace1a04c4a65903494f7e6d96cca, user_id=cf71a4aec63e4229a8bf9840dd5c5c51, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:44.040803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:44.055547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1247/2467] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:44 2022] PUT /identity/v3/projects/75d0ace1a04c4a65903494f7e6d96cca/users/cf71a4aec63e4229a8bf9840dd5c5c51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:44.059663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:44.069883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc1806-57cb-4aff-94fc-d6518d9fb142 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:44.130857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236f150-28e7-42a0-82b5-cf97be55e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:44.131557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236f150-28e7-42a0-82b5-cf97be55e56d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:44.131839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236f150-28e7-42a0-82b5-cf97be55e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:44.132145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236f150-28e7-42a0-82b5-cf97be55e56d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:44.132897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236f150-28e7-42a0-82b5-cf97be55e56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:44.134999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236f150-28e7-42a0-82b5-cf97be55e56d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:44.146196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1221/2468] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:44.150421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:44.160591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ef555-fb00-4727-bf4a-de5415e6aeb0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:44.165197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:44.166105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:44.166389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:44.166666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] PATH_INFO: `/v3/projects/75d0ace1a04c4a65903494f7e6d96cca/users/cf71a4aec63e4229a8bf9840dd5c5c51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:44.168594 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d0ace1a04c4a65903494f7e6d96cca, user_id=cf71a4aec63e4229a8bf9840dd5c5c51, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:44.171353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:44.182814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1248/2469] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:44 2022] PUT /identity/v3/projects/75d0ace1a04c4a65903494f7e6d96cca/users/cf71a4aec63e4229a8bf9840dd5c5c51/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:44.196821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879e664-870b-4189-837a-eec3f43b4ccb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:44.197115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879e664-870b-4189-837a-eec3f43b4ccb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:44.197398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879e664-870b-4189-837a-eec3f43b4ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:44.226481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d879e664-870b-4189-837a-eec3f43b4ccb None None] MFA Rules not processed for user `cf71a4aec63e4229a8bf9840dd5c5c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:44.227749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d879e664-870b-4189-837a-eec3f43b4ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:44.297658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1222/2470] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:44.309792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:44.311470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:44.368127 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5eccf-2a6d-4974-a08f-a9561da54bf6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:44.371777 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:44.372383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:44.372668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:44.372947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:44.385068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:44.386547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:44.397170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1249/2471] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:44.473502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879e664-870b-4189-837a-eec3f43b4ccb None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:44.484228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879e664-870b-4189-837a-eec3f43b4ccb None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:44.488074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3df342-46ad-4e88-b7ad-4bdb67a8d7ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:44.488745 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3df342-46ad-4e88-b7ad-4bdb67a8d7ee service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:44.489234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3df342-46ad-4e88-b7ad-4bdb67a8d7ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:44.489621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3df342-46ad-4e88-b7ad-4bdb67a8d7ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:44.512213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:44.558382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3df342-46ad-4e88-b7ad-4bdb67a8d7ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:44.559838 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3df342-46ad-4e88-b7ad-4bdb67a8d7ee service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:44.569895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1223/2472] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:52.647553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:52.649107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:52.680783 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:52.787894 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637324ab-98c9-4c47-8dea-4851127cfb38 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:52.793060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:52.794018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:52.794305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:52.794583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] PATH_INFO: `/v3/users/2f8c4f1f7e9944c8ae1b490c38eeeaa5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:52.795893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8c4f1f7e9944c8ae1b490c38eeeaa5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:52.798141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:52.851030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8c4f1f7e9944c8ae1b490c38eeeaa5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:52.851689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8c4f1f7e9944c8ae1b490c38eeeaa5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:52.852298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8c4f1f7e9944c8ae1b490c38eeeaa5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:52.860049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8c4f1f7e9944c8ae1b490c38eeeaa5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:52.869966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8c4f1f7e9944c8ae1b490c38eeeaa5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:52.887715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1250/2473] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:24:52 2022] DELETE /identity/v3/users/2f8c4f1f7e9944c8ae1b490c38eeeaa5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:52.894325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:52.894835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:52.894973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:52.921986 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:52.923194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:52.987205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1224/2474] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:52.996796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:52.998395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:53.042264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:53.073180 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbebeb6-d6ea-45fd-8faa-170bc7d07638 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.077451 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e3ca5-9a5f-4737-b972-9b5a6b410b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.078053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e3ca5-9a5f-4737-b972-9b5a6b410b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.078343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e3ca5-9a5f-4737-b972-9b5a6b410b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.078629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e3ca5-9a5f-4737-b972-9b5a6b410b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.089044 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e3ca5-9a5f-4737-b972-9b5a6b410b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.090619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e3ca5-9a5f-4737-b972-9b5a6b410b22 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.101617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1251/2475] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:53.368118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:53.369546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:53.389652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b726a-9356-4287-8f20-21e5c6a0a8b3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.394586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.395349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.395631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.395902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] PATH_INFO: `/v3/projects/12a1c871a00440e5ade5efea6883954a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.397082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a1c871a00440e5ade5efea6883954a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.399025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.446283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a1c871a00440e5ade5efea6883954a', 'request_id': 'req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:53.454566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a1c871a00440e5ade5efea6883954a', 'request_id': 'req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:53.461526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a1c871a00440e5ade5efea6883954a', 'request_id': 'req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:53.472808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] The token cache is being invalidate because project 12a1c871a00440e5ade5efea6883954a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:24:53.473094 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:53.473761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-d8db5a45-f50c-42e6-ad0f-265dbe0727d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:53.475974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1225/2476] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:24:53 2022] DELETE /identity/v3/projects/12a1c871a00440e5ade5efea6883954a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:53.480885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493843-a759-48b4-b376-7e844bebcbb2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.481187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493843-a759-48b4-b376-7e844bebcbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.481470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a493843-a759-48b4-b376-7e844bebcbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.520339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1a493843-a759-48b4-b376-7e844bebcbb2 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:53.521553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1a493843-a759-48b4-b376-7e844bebcbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:53.578378 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1252/2477] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:24:53.587434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d623-8a8f-4c8d-96a1-ef95edda5947 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.587730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d623-8a8f-4c8d-96a1-ef95edda5947 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.588015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637d623-8a8f-4c8d-96a1-ef95edda5947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.606271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8637d623-8a8f-4c8d-96a1-ef95edda5947 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:53.607464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8637d623-8a8f-4c8d-96a1-ef95edda5947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:53.615818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1226/2478] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:53.619842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a493843-a759-48b4-b376-7e844bebcbb2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:53.630152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a493843-a759-48b4-b376-7e844bebcbb2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.634746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.635400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.635688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.635970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.636694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:24:53.637008 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.638652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.650600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1253/2479] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:24:53.654549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637d623-8a8f-4c8d-96a1-ef95edda5947 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:53.663744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637d623-8a8f-4c8d-96a1-ef95edda5947 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.668365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.669040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.669326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.669607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.670336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.672245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.697921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1227/2480] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:53 2022] POST /identity/v3/projects => generated 405 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:53.702001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:53.711952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2308a2-890f-4d8c-a26a-ca34a4f7e9fc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.766775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.767942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.768228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.768507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.769182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.771203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.791187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1254/2481] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:53 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:53.795194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:53.805557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9101d5f-ab02-41b9-947d-7ad73992e8f3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.810477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.811131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.811415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.811694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.812405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.814044 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.815348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:53.826597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1228/2482] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:53.830929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:53.840726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da080c-4fa9-4e3c-96de-fae397b6f223 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.846112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2a145-817b-48b7-be7b-13ed45a6e14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.847079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2a145-817b-48b7-be7b-13ed45a6e14f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.847367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2a145-817b-48b7-be7b-13ed45a6e14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.847651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2a145-817b-48b7-be7b-13ed45a6e14f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.848375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2a145-817b-48b7-be7b-13ed45a6e14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.850026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2a145-817b-48b7-be7b-13ed45a6e14f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.859956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1255/2483] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:53.864358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:53.874032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a425b-8a1d-4aa1-9b03-a7f985ee742f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:53.879698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:53.880565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.880845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.881119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] PATH_INFO: `/v3/projects/48b557fcf7364da681acf16b93ca8dcf/users/b1e486e64c3748e382146699439cfd68/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.899608 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b557fcf7364da681acf16b93ca8dcf, user_id=b1e486e64c3748e382146699439cfd68, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:53.902328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:53.915552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1229/2484] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:53 2022] PUT /identity/v3/projects/48b557fcf7364da681acf16b93ca8dcf/users/b1e486e64c3748e382146699439cfd68/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:53.925848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d86f9-70ed-4b48-beeb-0006bb66617c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:53.926165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d86f9-70ed-4b48-beeb-0006bb66617c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:53.926445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d86f9-70ed-4b48-beeb-0006bb66617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:53.952520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-560d86f9-70ed-4b48-beeb-0006bb66617c None None] MFA Rules not processed for user `b1e486e64c3748e382146699439cfd68`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:53.953763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-560d86f9-70ed-4b48-beeb-0006bb66617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:53.960351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:54.028891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1256/2485] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:54.033324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.102186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c6491-ac0d-4b9a-b668-e71afa289b58 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.106843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.107505 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.107793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.108073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.108807 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.110798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.147884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1230/2486] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:54 2022] POST /identity/v3/projects => generated 405 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:54.149183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d86f9-70ed-4b48-beeb-0006bb66617c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.159453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d86f9-70ed-4b48-beeb-0006bb66617c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.225665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.226409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.226695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.226972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.227792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.230089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.257509 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1257/2487] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:54 2022] POST /identity/v3/users => generated 309 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:54.258691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.269371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e66fa-0d3b-488d-8ee7-19faf292bcf3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.274217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.274881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.275168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.275564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.276278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.277881 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.289657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1231/2488] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:24:54.293834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.305375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2cde3-efad-406a-92dc-7e0c3f8901c7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.311605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.312730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.313084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.313367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] PATH_INFO: `/v3/projects/d9960a5f34b844d39c1359f0b840497b/users/5f2ae03dbbce46ecb80d2f033df4695c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.328281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9960a5f34b844d39c1359f0b840497b, user_id=5f2ae03dbbce46ecb80d2f033df4695c, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.331579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.344672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1258/2489] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:54 2022] PUT /identity/v3/projects/d9960a5f34b844d39c1359f0b840497b/users/5f2ae03dbbce46ecb80d2f033df4695c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:54.348972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.362232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0779e3-45e2-444c-a755-f72d0688c39b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.421006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707ba3a-4778-4596-8606-f5e9f4dff534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.421798 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ba3a-4778-4596-8606-f5e9f4dff534 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.422091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ba3a-4778-4596-8606-f5e9f4dff534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.422454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707ba3a-4778-4596-8606-f5e9f4dff534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.423188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707ba3a-4778-4596-8606-f5e9f4dff534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.425417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707ba3a-4778-4596-8606-f5e9f4dff534 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.435655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1232/2490] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:54.440376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.452068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636727a-d4c8-47e2-8974-84bb3c8f8283 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.458020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.458899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.459184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.460038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] PATH_INFO: `/v3/projects/d9960a5f34b844d39c1359f0b840497b/users/5f2ae03dbbce46ecb80d2f033df4695c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.462024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9960a5f34b844d39c1359f0b840497b, user_id=5f2ae03dbbce46ecb80d2f033df4695c, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.464686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.476411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1259/2491] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:54 2022] PUT /identity/v3/projects/d9960a5f34b844d39c1359f0b840497b/users/5f2ae03dbbce46ecb80d2f033df4695c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:54.488976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ca4b0-5c9c-47bb-8bc2-5d31212e8098 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.489268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ca4b0-5c9c-47bb-8bc2-5d31212e8098 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.489595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ca4b0-5c9c-47bb-8bc2-5d31212e8098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.518314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-534ca4b0-5c9c-47bb-8bc2-5d31212e8098 None None] MFA Rules not processed for user `5f2ae03dbbce46ecb80d2f033df4695c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:54.519519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-534ca4b0-5c9c-47bb-8bc2-5d31212e8098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:54.552059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:54.591911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1233/2492] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:54.602910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.604597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:54.665500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5f35-2264-4bb8-b61d-0297d695a40a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.669262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e7c00-87e8-47a4-8381-e4046fa76b16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.669922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e7c00-87e8-47a4-8381-e4046fa76b16 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.670217 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e7c00-87e8-47a4-8381-e4046fa76b16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.670492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e7c00-87e8-47a4-8381-e4046fa76b16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.679669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e7c00-87e8-47a4-8381-e4046fa76b16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.681112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e7c00-87e8-47a4-8381-e4046fa76b16 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.691368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1260/2493] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:54 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:54.837007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ca4b0-5c9c-47bb-8bc2-5d31212e8098 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:54.900656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534ca4b0-5c9c-47bb-8bc2-5d31212e8098 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:54.905388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:54.906325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.906606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.906880 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] PATH_INFO: `/v3/users/b1e486e64c3748e382146699439cfd68` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:54.908632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e486e64c3748e382146699439cfd68)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:54.910577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:54.955870 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e486e64c3748e382146699439cfd68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:54.956521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e486e64c3748e382146699439cfd68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:54.963311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e486e64c3748e382146699439cfd68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:54.963937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e486e64c3748e382146699439cfd68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:54.971195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e486e64c3748e382146699439cfd68'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:54.989694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1234/2494] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:24:54 2022] DELETE /identity/v3/users/b1e486e64c3748e382146699439cfd68 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:54.994237 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:54.994532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:54.994817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:55.000453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:55.013871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:55.015069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:55.072881 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1261/2495] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:55.083450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:55.084823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:55.138099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:55.145144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deea8ce-ce95-4a00-a5df-c05e22516e03 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:55.149943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:55.150604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:55.150884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:55.151161 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:55.160266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:55.161688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:55.171057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1235/2496] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:55.451431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:55.455984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:55.456123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:55.457614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:55.468478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77efb2-7224-4bd4-a130-b1bffbaebad9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:55.473500 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:55.474403 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:55.474690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:55.474971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] PATH_INFO: `/v3/projects/48b557fcf7364da681acf16b93ca8dcf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:55.476168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b557fcf7364da681acf16b93ca8dcf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:55.478166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:55.479023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcd55d-2673-424d-868c-35fb01e8fe38 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:55.484639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:55.485580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:55.485995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:55.486342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] PATH_INFO: `/v3/users/f8e1943fd89a40dc9d843431e36512b1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:55.487592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e1943fd89a40dc9d843431e36512b1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:55.489583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:55.530954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b557fcf7364da681acf16b93ca8dcf', 'request_id': 'req-238c1baf-b18e-4250-940e-d4c8fd66c0cc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.539492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b557fcf7364da681acf16b93ca8dcf', 'request_id': 'req-238c1baf-b18e-4250-940e-d4c8fd66c0cc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.540968 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e1943fd89a40dc9d843431e36512b1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.541664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e1943fd89a40dc9d843431e36512b1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.545897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b557fcf7364da681acf16b93ca8dcf', 'request_id': 'req-238c1baf-b18e-4250-940e-d4c8fd66c0cc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.549110 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e1943fd89a40dc9d843431e36512b1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.549800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e1943fd89a40dc9d843431e36512b1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.556344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] The token cache is being invalidate because project 48b557fcf7364da681acf16b93ca8dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:24:55.556632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.557267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.558146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab2383bf-7a5a-4deb-8728-82cdcdce563c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e1943fd89a40dc9d843431e36512b1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.558920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1262/2497] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:24:55 2022] DELETE /identity/v3/projects/48b557fcf7364da681acf16b93ca8dcf => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:55.562808 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:55.564666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:55.577749 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1236/2498] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:24:55 2022] DELETE /identity/v3/users/f8e1943fd89a40dc9d843431e36512b1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:55.582975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:55.583556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:55.583556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:55.601063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:55.602307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:55.630559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c1baf-b18e-4250-940e-d4c8fd66c0cc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:55.661868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:55.663431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1237/2499] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:55.673098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:55.675427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:55.691289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:55.692173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:55.692461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:55.692743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] PATH_INFO: `/v3/users/5f2ae03dbbce46ecb80d2f033df4695c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:55.693975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f2ae03dbbce46ecb80d2f033df4695c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:55.695906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:55.733685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf22aec-d304-4143-a0fc-6ce29a512981 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:55.737357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:55.738039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:55.738326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:55.738618 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:55.743555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f2ae03dbbce46ecb80d2f033df4695c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.744236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f2ae03dbbce46ecb80d2f033df4695c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.744892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f2ae03dbbce46ecb80d2f033df4695c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.747970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:55.749670 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:55.754271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f2ae03dbbce46ecb80d2f033df4695c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.761121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1238/2500] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:24:55.761604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f2ae03dbbce46ecb80d2f033df4695c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:55.780328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1263/2501] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:24:55 2022] DELETE /identity/v3/users/5f2ae03dbbce46ecb80d2f033df4695c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:56.037958 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.039597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.055990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:56.060481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b3ab8-63c7-4c1e-bf70-9f2e19c7fdf3 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.076159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.077920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.143463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.144042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.144356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.144631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] PATH_INFO: `/v3/projects/c6db296a114e4480aa581d24d5192a41` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.146131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6db296a114e4480aa581d24d5192a41)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.147832 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.159722 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9d024-3c9b-40b4-8119-bd2804b7e3b4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.167875 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.168959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.169421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.169916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] PATH_INFO: `/v3/projects/d9960a5f34b844d39c1359f0b840497b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.171309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9960a5f34b844d39c1359f0b840497b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.174036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.202462 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6db296a114e4480aa581d24d5192a41', 'request_id': 'req-43ffe595-1f4c-403d-a191-fbf578462bec'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.212582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6db296a114e4480aa581d24d5192a41', 'request_id': 'req-43ffe595-1f4c-403d-a191-fbf578462bec'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.219877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6db296a114e4480aa581d24d5192a41', 'request_id': 'req-43ffe595-1f4c-403d-a191-fbf578462bec'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.231652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9960a5f34b844d39c1359f0b840497b', 'request_id': 'req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.232771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] The token cache is being invalidate because project c6db296a114e4480aa581d24d5192a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:24:56.233121 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.233973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.236149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1239/2502] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:24:56 2022] DELETE /identity/v3/projects/c6db296a114e4480aa581d24d5192a41 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:56.236824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:56.240629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.242148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.242929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9960a5f34b844d39c1359f0b840497b', 'request_id': 'req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.252544 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9960a5f34b844d39c1359f0b840497b', 'request_id': 'req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.264795 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] The token cache is being invalidate because project d9960a5f34b844d39c1359f0b840497b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:24:56.265196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.268072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-eaadca17-85d4-4f7d-bec2-2f3731c08e0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.270571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1264/2503] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:24:56 2022] DELETE /identity/v3/projects/d9960a5f34b844d39c1359f0b840497b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:56.275152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebf72-c906-4f56-b8c8-2af0be63f766 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.275520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebf72-c906-4f56-b8c8-2af0be63f766 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.276025 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cebf72-c906-4f56-b8c8-2af0be63f766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.295525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-62cebf72-c906-4f56-b8c8-2af0be63f766 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:56.296744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62cebf72-c906-4f56-b8c8-2af0be63f766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.321432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffe595-1f4c-403d-a191-fbf578462bec admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.362799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1265/2504] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:56.370248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af544850-f090-471d-8b56-266c4d3dbbdf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.370604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af544850-f090-471d-8b56-266c4d3dbbdf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.370976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af544850-f090-471d-8b56-266c4d3dbbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.371278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:56.389979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-af544850-f090-471d-8b56-266c4d3dbbdf None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:56.391197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af544850-f090-471d-8b56-266c4d3dbbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.391528 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.392456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.392741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.393073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] PATH_INFO: `/v3/users/cf71a4aec63e4229a8bf9840dd5c5c51` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.394301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf71a4aec63e4229a8bf9840dd5c5c51)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.396163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.399202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1266/2505] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:24:56.403279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af544850-f090-471d-8b56-266c4d3dbbdf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.414542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af544850-f090-471d-8b56-266c4d3dbbdf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.419165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.419821 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.420110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.420392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.421171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:24:56.421556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.424196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.437302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1267/2506] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:24:56.441493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.448685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf71a4aec63e4229a8bf9840dd5c5c51'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.449354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf71a4aec63e4229a8bf9840dd5c5c51'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.451197 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c36f9-01d4-47bb-a750-95567efe72a6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.455785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.456273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf71a4aec63e4229a8bf9840dd5c5c51'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.456410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.456694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.456922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf71a4aec63e4229a8bf9840dd5c5c51'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.457021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.457685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.459699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.464500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf71a4aec63e4229a8bf9840dd5c5c51'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.486535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1268/2507] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:56.487247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1240/2508] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:24:56 2022] DELETE /identity/v3/users/cf71a4aec63e4229a8bf9840dd5c5c51 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:56.490556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.492303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.559285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabf105-3141-42aa-bd49-4d1275033a29 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.605938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:56.620393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.621067 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.621352 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.621628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.622346 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.624288 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.649749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1269/2509] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:56.652154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.662770 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a8ad2-caeb-4ab3-9ac3-ccd908ceddf4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.667438 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.668084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.668368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.668650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.669397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.671035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.685508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1241/2510] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:56.686482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.696837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24627f8-b9fd-46b2-8ad2-c056bd65861e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.701461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.702154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.702437 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.702720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.703441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.705571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.715609 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1270/2511] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:24:56.720131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.730763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d0ed3-57a9-41f3-965d-cad00d058778 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.735375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb2583-eaab-406f-8466-388476e9d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.736771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb2583-eaab-406f-8466-388476e9d963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.737053 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb2583-eaab-406f-8466-388476e9d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.737327 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb2583-eaab-406f-8466-388476e9d963 admin admin] PATH_INFO: `/v3/projects/efff379132804d6b9d3fa0ee8ed5a310/users/03d3d6d7616f4c71bed97155d11e1008/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.754017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb2583-eaab-406f-8466-388476e9d963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efff379132804d6b9d3fa0ee8ed5a310, user_id=03d3d6d7616f4c71bed97155d11e1008, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.756709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb2583-eaab-406f-8466-388476e9d963 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.772322 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1242/2512] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:56 2022] PUT /identity/v3/projects/efff379132804d6b9d3fa0ee8ed5a310/users/03d3d6d7616f4c71bed97155d11e1008/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:56.773374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.775236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.781132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ff52a-4710-4475-9e19-96cf70ea657f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.781522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ff52a-4710-4475-9e19-96cf70ea657f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.781856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ff52a-4710-4475-9e19-96cf70ea657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.788796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251d2bf-abed-4819-bbe7-af2ad690043b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.812097 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-060ff52a-4710-4475-9e19-96cf70ea657f None None] MFA Rules not processed for user `03d3d6d7616f4c71bed97155d11e1008`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:56.813500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-060ff52a-4710-4475-9e19-96cf70ea657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.855376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.856246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.856540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.856820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] PATH_INFO: `/v3/projects/75d0ace1a04c4a65903494f7e6d96cca` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.858047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=75d0ace1a04c4a65903494f7e6d96cca)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.860010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.880191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1243/2513] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:56.884860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ff52a-4710-4475-9e19-96cf70ea657f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:56.896303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ff52a-4710-4475-9e19-96cf70ea657f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:56.908186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:56.908353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75d0ace1a04c4a65903494f7e6d96cca', 'request_id': 'req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.918172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75d0ace1a04c4a65903494f7e6d96cca', 'request_id': 'req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.926005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75d0ace1a04c4a65903494f7e6d96cca', 'request_id': 'req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.937206 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] The token cache is being invalidate because project 75d0ace1a04c4a65903494f7e6d96cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:24:56.937501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.938202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e294cfb1-8da4-42ed-bddd-5d4ff721cd1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:24:56.940136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1271/2514] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:24:56 2022] DELETE /identity/v3/projects/75d0ace1a04c4a65903494f7e6d96cca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:56.945095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676ba07-66a3-4add-8144-c89b6ba95191 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.945387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676ba07-66a3-4add-8144-c89b6ba95191 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.945671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676ba07-66a3-4add-8144-c89b6ba95191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.960223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:56.960916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.961201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.961523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.962275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:56.964272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:56.965478 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1676ba07-66a3-4add-8144-c89b6ba95191 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:56.966787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1676ba07-66a3-4add-8144-c89b6ba95191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:56.975174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1272/2515] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:24:56.982895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250906d-0933-4dc0-907c-a19fbeb11d9b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:56.983211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250906d-0933-4dc0-907c-a19fbeb11d9b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:56.983493 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250906d-0933-4dc0-907c-a19fbeb11d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:56.997616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1244/2516] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:57.002195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.005015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:57.007344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0250906d-0933-4dc0-907c-a19fbeb11d9b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:57.008605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0250906d-0933-4dc0-907c-a19fbeb11d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:57.053439 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:57.070151 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1273/2517] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:24:57.071602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211c1f-8f2a-47d6-9a0e-131c3e5eee19 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.074460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250906d-0933-4dc0-907c-a19fbeb11d9b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.077635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.078483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.078766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.079039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.079861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.081985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.086274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250906d-0933-4dc0-907c-a19fbeb11d9b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.090999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.091828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.092241 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.092680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.093553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:24:57.093952 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.096400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.106357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1245/2518] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:57.107369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.112694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1274/2519] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:24:57.116769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.117946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8a773-9299-4a4d-a05f-15b983e933d2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.122488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.123132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.123416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.123689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.124401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.126028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.126822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bec35-7ce3-4df7-9d1e-2f24f5e19967 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.139750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.139951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1246/2520] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:24:57.140417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.140729 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.140827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.140960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.141657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.143634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.151016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4eff2-d250-4923-8c65-cc7edabb334f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.155794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.156681 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.156960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.157273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] PATH_INFO: `/v3/projects/46ae6683485d486e908910018b968f45/users/cb92288822bc48e9b2b65a7eaa936de5/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.170645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1275/2521] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:57.173010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ae6683485d486e908910018b968f45, user_id=cb92288822bc48e9b2b65a7eaa936de5, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.175377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.176450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.189589 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1247/2522] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:57 2022] PUT /identity/v3/projects/46ae6683485d486e908910018b968f45/users/cb92288822bc48e9b2b65a7eaa936de5/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:57.193529 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.237847 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08645d7-1ad3-4655-8e86-1e39fddcf66b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.250169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:57.256510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e54f-255f-4e46-bdde-e01eb5647c8e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.261425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.262127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.262414 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.262691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.263406 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.265007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.276217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1248/2523] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:24:57.280423 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.290369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f69334-d8aa-4ef5-ab2b-290b330eded7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.295562 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.296401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d65c5-ca97-4e1a-965d-2565bdb83752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.296557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.296712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.296996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] PATH_INFO: `/v3/projects/46ae6683485d486e908910018b968f45/users/cb92288822bc48e9b2b65a7eaa936de5/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.297124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d65c5-ca97-4e1a-965d-2565bdb83752 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.297343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d65c5-ca97-4e1a-965d-2565bdb83752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.297634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d65c5-ca97-4e1a-965d-2565bdb83752 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.298427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d65c5-ca97-4e1a-965d-2565bdb83752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.299119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ae6683485d486e908910018b968f45, user_id=cb92288822bc48e9b2b65a7eaa936de5, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.300305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d65c5-ca97-4e1a-965d-2565bdb83752 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.301862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.312199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1249/2524] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:57 2022] PUT /identity/v3/projects/46ae6683485d486e908910018b968f45/users/cb92288822bc48e9b2b65a7eaa936de5/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:57.322011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1276/2525] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:57.326006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.327148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.327447 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.327735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.355062 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] MFA Rules not processed for user `cb92288822bc48e9b2b65a7eaa936de5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:57.356262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:57.394477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8172ea-6d6b-4e15-942a-daf73147177e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.397771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:57.399465 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.400158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.400432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.400706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.401482 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.403047 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.413479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1250/2526] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:57.417043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.427226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1277/2527] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:57.433458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab651bc-6006-46e5-a954-3479e9718939 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.444655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.445332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.445614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.446000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.446725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.447052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.448745 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:57.448878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.459489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1251/2528] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:24:57.463534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.473113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9acde-44fc-4bc9-b119-56a7049f347d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.477700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee82617-fd6e-4261-976c-6c860fd20994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.478611 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82617-fd6e-4261-976c-6c860fd20994 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.478893 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82617-fd6e-4261-976c-6c860fd20994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.479168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee82617-fd6e-4261-976c-6c860fd20994 admin admin] PATH_INFO: `/v3/projects/c3a2831b884d4348a5bded6fae5d5796/users/fe591c733e224cccbaac77e5050b0851/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.494952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee82617-fd6e-4261-976c-6c860fd20994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a2831b884d4348a5bded6fae5d5796, user_id=fe591c733e224cccbaac77e5050b0851, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.497744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee82617-fd6e-4261-976c-6c860fd20994 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.506601 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c9312-ba5f-4eba-8093-3e1d41b8711d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.512036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1252/2529] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:57 2022] PUT /identity/v3/projects/c3a2831b884d4348a5bded6fae5d5796/users/fe591c733e224cccbaac77e5050b0851/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:57.522575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc759b-7ef1-4869-af44-4242c45a1b71 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.522950 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc759b-7ef1-4869-af44-4242c45a1b71 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.523264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc759b-7ef1-4869-af44-4242c45a1b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.548864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-effc759b-7ef1-4869-af44-4242c45a1b71 None None] MFA Rules not processed for user `fe591c733e224cccbaac77e5050b0851`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:57.550099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-effc759b-7ef1-4869-af44-4242c45a1b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:57.556699 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.557351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.557640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.557988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.617368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1253/2530] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:24:57.618832 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.620323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.621957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc759b-7ef1-4869-af44-4242c45a1b71 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.630434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1278/2531] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:24:57.653957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:57.678426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.679811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:57.688413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc759b-7ef1-4869-af44-4242c45a1b71 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.691168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901ee7c-ad2e-4578-8d7d-4a467c492986 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.693148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.693849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.694138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.694421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.694931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.695111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.695581 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.695872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.696154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.697048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.697991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:57.724523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1254/2532] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:57.728664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.743276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:57.764282 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.765756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.775473 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1279/2533] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:24:57.790710 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c40f-6663-4b41-93f6-8fab8705b8f1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.795771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.796490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.796775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.797071 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.797814 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.799911 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.821318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1255/2534] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:57.825568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.836160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8a440-144b-45a9-8d57-60f80ed105cc service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.840909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.841565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.841882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.842160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.843019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.844679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.855607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1280/2535] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:57.859753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.870335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8d57e-6356-460c-8c9e-49f92d92e042 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.875412 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f121ff1-e498-4b88-af61-f4b8a2ae766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.876074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f121ff1-e498-4b88-af61-f4b8a2ae766e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.876857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f121ff1-e498-4b88-af61-f4b8a2ae766e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.877137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f121ff1-e498-4b88-af61-f4b8a2ae766e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.877890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f121ff1-e498-4b88-af61-f4b8a2ae766e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.879498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f121ff1-e498-4b88-af61-f4b8a2ae766e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.890454 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1256/2536] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:57.895207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:57.905278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceea54-01d1-406f-81c2-eeb19c31d2c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:57.910972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:57.911955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.912245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.912532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] PATH_INFO: `/v3/projects/ef040ca2d7674e25a5a9e047821b8ff3/users/3705454e8fd74010b7d3633e98a145c8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.929827 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef040ca2d7674e25a5a9e047821b8ff3, user_id=3705454e8fd74010b7d3633e98a145c8, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:57.933229 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:57.948863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1281/2537] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:57 2022] PUT /identity/v3/projects/ef040ca2d7674e25a5a9e047821b8ff3/users/3705454e8fd74010b7d3633e98a145c8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:57.960347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb6ca8-8281-44c6-aec0-236744b450d2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:57.960699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb6ca8-8281-44c6-aec0-236744b450d2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:57.960988 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb6ca8-8281-44c6-aec0-236744b450d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:57.992444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e0bb6ca8-8281-44c6-aec0-236744b450d2 None None] MFA Rules not processed for user `3705454e8fd74010b7d3633e98a145c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:57.994029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0bb6ca8-8281-44c6-aec0-236744b450d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:58.082450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1257/2538] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:58.087286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:58.153951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1eb5f-55a6-4056-9deb-3bc21e144b10 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:58.158785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:58.159388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.159675 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.159955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.160668 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:58.162636 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:58.188153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1282/2539] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:24:58 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:24:58.192809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb6ca8-8281-44c6-aec0-236744b450d2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:58.203344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb6ca8-8281-44c6-aec0-236744b450d2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:58.222042 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:58.261764 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:58.262489 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.262772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.263046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.263720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:58.265658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:58.287973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1258/2540] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:24:58 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:24:58.292014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:58.302177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61ccee-c937-4964-9402-56d7976ca818 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:58.306909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:58.307565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.307852 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.308132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.308856 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:58.310667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:58.321474 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1283/2541] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:58.325644 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:58.336159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c935c2-f046-4d8d-9815-b1c4ff34f40c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:58.340668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:58.341537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.341845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.342124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] PATH_INFO: `/v3/projects/e4455d03c1f6414f963c4473c374149c/users/d5fa7720f3074fdaa0ac5d9b7c469892/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.357383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4455d03c1f6414f963c4473c374149c, user_id=d5fa7720f3074fdaa0ac5d9b7c469892, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:58.360059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:58.373444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1259/2542] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:58 2022] PUT /identity/v3/projects/e4455d03c1f6414f963c4473c374149c/users/d5fa7720f3074fdaa0ac5d9b7c469892/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:24:58.377902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:58.388227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5a56d-17e6-4c20-983c-8ab0165d0a95 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:58.444674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97e975-ca22-4e03-94f8-737565959ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:58.445256 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e975-ca22-4e03-94f8-737565959ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.445541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e975-ca22-4e03-94f8-737565959ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.445849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97e975-ca22-4e03-94f8-737565959ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.446589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97e975-ca22-4e03-94f8-737565959ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:58.448182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97e975-ca22-4e03-94f8-737565959ad8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:58.458163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1284/2543] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:24:58.462250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:58.473160 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00abf3fc-13b5-421b-ae3a-66cb8dffeb7b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:58.477628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:58.478535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.478844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.479125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] PATH_INFO: `/v3/projects/e4455d03c1f6414f963c4473c374149c/users/d5fa7720f3074fdaa0ac5d9b7c469892/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.481082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4455d03c1f6414f963c4473c374149c, user_id=d5fa7720f3074fdaa0ac5d9b7c469892, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:58.483758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:58.494330 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1260/2544] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:24:58 2022] PUT /identity/v3/projects/e4455d03c1f6414f963c4473c374149c/users/d5fa7720f3074fdaa0ac5d9b7c469892/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:24:58.504122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a723ca5-17b7-466b-bb0c-6c84f27cbf36 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.504415 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a723ca5-17b7-466b-bb0c-6c84f27cbf36 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.504700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a723ca5-17b7-466b-bb0c-6c84f27cbf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.529774 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3a723ca5-17b7-466b-bb0c-6c84f27cbf36 None None] MFA Rules not processed for user `d5fa7720f3074fdaa0ac5d9b7c469892`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:24:58.530967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a723ca5-17b7-466b-bb0c-6c84f27cbf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:24:58.536398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:24:58.599775 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1285/2545] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:24:58.610974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:58.664872 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf30f36-9556-46c2-9839-cb010ba85bbe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:58.668594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:58.669263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:58.669554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:58.669867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:58.680238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:58.681890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:58.691810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1261/2546] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:24:59.218214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a723ca5-17b7-466b-bb0c-6c84f27cbf36 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:24:59.232056 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a723ca5-17b7-466b-bb0c-6c84f27cbf36 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:24:59.233161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7101b-443e-435e-9f1a-5547f98e2b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:24:59.233848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7101b-443e-435e-9f1a-5547f98e2b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:24:59.234138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7101b-443e-435e-9f1a-5547f98e2b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:24:59.234423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7101b-443e-435e-9f1a-5547f98e2b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:24:59.291472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7101b-443e-435e-9f1a-5547f98e2b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:24:59.292919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7101b-443e-435e-9f1a-5547f98e2b6e service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:24:59.304750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1286/2547] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:24:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:01.085259 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:01.153147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934a2b8-f759-4ac2-9696-6103a9f16591 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:01.157682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:01.158594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:01.158885 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:01.159166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] PATH_INFO: `/v3/users/03d3d6d7616f4c71bed97155d11e1008` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:01.160357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d3d6d7616f4c71bed97155d11e1008)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:01.162344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:01.183852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:01.210839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d3d6d7616f4c71bed97155d11e1008'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.211442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d3d6d7616f4c71bed97155d11e1008'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.218179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d3d6d7616f4c71bed97155d11e1008'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.218822 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d3d6d7616f4c71bed97155d11e1008'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.226910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d3d6d7616f4c71bed97155d11e1008'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.244238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1262/2548] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:01 2022] DELETE /identity/v3/users/03d3d6d7616f4c71bed97155d11e1008 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:01.248769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:01.249257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:01.249356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:01.266649 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:01.267896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:01.326679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1287/2549] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:01.336862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:01.338266 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:01.396514 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e6b51-e70e-4400-957d-300a44035d0b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:01.400131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:01.400799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:01.401072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:01.401350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:01.410965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:01.412393 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:01.421851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1263/2550] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:01.693635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:01.695272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:01.710092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14241a-6a57-4acb-acdd-e755e81e2b0b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:01.715257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:01.716138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:01.716472 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:01.716749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] PATH_INFO: `/v3/projects/efff379132804d6b9d3fa0ee8ed5a310` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:01.717955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efff379132804d6b9d3fa0ee8ed5a310)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:01.719896 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:01.758713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:01.768255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efff379132804d6b9d3fa0ee8ed5a310', 'request_id': 'req-5ea41ead-55f0-4bf9-981d-555dfcba3d86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.775408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efff379132804d6b9d3fa0ee8ed5a310', 'request_id': 'req-5ea41ead-55f0-4bf9-981d-555dfcba3d86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.781409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efff379132804d6b9d3fa0ee8ed5a310', 'request_id': 'req-5ea41ead-55f0-4bf9-981d-555dfcba3d86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.791598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] The token cache is being invalidate because project efff379132804d6b9d3fa0ee8ed5a310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:01.791884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.792512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.794081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1288/2551] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:01 2022] DELETE /identity/v3/projects/efff379132804d6b9d3fa0ee8ed5a310 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:01.798602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:01.808258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc18143-3ee8-4e9f-a17b-10099a40d5da service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:01.864399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:01.865282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:01.865567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:01.865890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] PATH_INFO: `/v3/users/cb92288822bc48e9b2b65a7eaa936de5` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:01.867079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb92288822bc48e9b2b65a7eaa936de5)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:01.869001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:01.912488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:01.919286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb92288822bc48e9b2b65a7eaa936de5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.920156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb92288822bc48e9b2b65a7eaa936de5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.928816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb92288822bc48e9b2b65a7eaa936de5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.929691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb92288822bc48e9b2b65a7eaa936de5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.938332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c39e8987-6bb0-45e8-9564-21e9dc7e74e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb92288822bc48e9b2b65a7eaa936de5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:01.957131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1264/2552] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:01 2022] DELETE /identity/v3/users/cb92288822bc48e9b2b65a7eaa936de5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:02.249163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.250692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:02.268102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea41ead-55f0-4bf9-981d-555dfcba3d86 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.327590 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.328448 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.328727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.329003 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] PATH_INFO: `/v3/projects/46ae6683485d486e908910018b968f45` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.330810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ae6683485d486e908910018b968f45)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.332750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.379947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ae6683485d486e908910018b968f45', 'request_id': 'req-4feefb28-1be5-4953-82ac-24c0f3c0806a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:02.387222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ae6683485d486e908910018b968f45', 'request_id': 'req-4feefb28-1be5-4953-82ac-24c0f3c0806a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:02.393570 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ae6683485d486e908910018b968f45', 'request_id': 'req-4feefb28-1be5-4953-82ac-24c0f3c0806a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:02.403956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] The token cache is being invalidate because project 46ae6683485d486e908910018b968f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:02.404247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:02.404947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4feefb28-1be5-4953-82ac-24c0f3c0806a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:02.406602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1289/2553] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:02 2022] DELETE /identity/v3/projects/46ae6683485d486e908910018b968f45 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:02.411659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f495ad-0d54-44b7-9b7b-231a02c57ecc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.412132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f495ad-0d54-44b7-9b7b-231a02c57ecc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.412261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f495ad-0d54-44b7-9b7b-231a02c57ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.430243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-43f495ad-0d54-44b7-9b7b-231a02c57ecc None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:02.431450 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-43f495ad-0d54-44b7-9b7b-231a02c57ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:02.490507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1265/2554] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:02.498796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3345ea-fb41-4220-98c7-6619efc2a2a3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.499135 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3345ea-fb41-4220-98c7-6619efc2a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.499456 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3345ea-fb41-4220-98c7-6619efc2a2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.518823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6c3345ea-fb41-4220-98c7-6619efc2a2a3 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:02.520024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c3345ea-fb41-4220-98c7-6619efc2a2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:02.527912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1290/2555] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:02.532261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f495ad-0d54-44b7-9b7b-231a02c57ecc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.542052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f495ad-0d54-44b7-9b7b-231a02c57ecc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.546534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.547186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.547466 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.547743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.548956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:02.549380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.551385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.562647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1266/2556] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:02.567543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3345ea-fb41-4220-98c7-6619efc2a2a3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.576946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3345ea-fb41-4220-98c7-6619efc2a2a3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.581576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.582253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.582532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.582806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.583498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.585396 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.611149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1291/2557] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:02 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:02.615168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.624909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33abe9-17ad-4cfb-8acd-3fbaed4c08af admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.680616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.681301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.681583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.681894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.682586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.684520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.705072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1267/2558] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:02 2022] POST /identity/v3/users => generated 305 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:02.709204 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.719665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22a706-7146-4678-ab34-0f27fb0802df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.724721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.725375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.725664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.725984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.726740 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.728372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.739863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1292/2559] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:02.743759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.754051 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3173cb-65e1-477e-b48d-6b2968314fea admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.758627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64272b5f-5898-4c2c-8831-ae7b7d54d641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.759271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272b5f-5898-4c2c-8831-ae7b7d54d641 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.759553 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272b5f-5898-4c2c-8831-ae7b7d54d641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.759826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64272b5f-5898-4c2c-8831-ae7b7d54d641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.760537 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64272b5f-5898-4c2c-8831-ae7b7d54d641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.762148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64272b5f-5898-4c2c-8831-ae7b7d54d641 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.772444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1268/2560] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:02.776306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.786164 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc08935-ce37-46b6-b2cf-148cc0dcf694 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:02.790655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:02.791805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.792250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.792535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] PATH_INFO: `/v3/projects/03b94ea091b94ce99cd7495b6682f0b1/users/0b5b4df3ceea413cab57f196e5df36a0/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.804212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:02.807838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b94ea091b94ce99cd7495b6682f0b1, user_id=0b5b4df3ceea413cab57f196e5df36a0, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:02.811152 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:02.826307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1293/2561] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:02 2022] PUT /identity/v3/projects/03b94ea091b94ce99cd7495b6682f0b1/users/0b5b4df3ceea413cab57f196e5df36a0/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:02.837015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09133105-7bcd-409a-8222-6923a78009db None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:02.837329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09133105-7bcd-409a-8222-6923a78009db None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:02.837609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09133105-7bcd-409a-8222-6923a78009db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:02.863521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-09133105-7bcd-409a-8222-6923a78009db None None] MFA Rules not processed for user `0b5b4df3ceea413cab57f196e5df36a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:02.864705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-09133105-7bcd-409a-8222-6923a78009db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:02.876581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:02.929777 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1269/2562] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:02.934235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:02.997002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5ace5-c0ca-4dce-a303-de7bb056bbcf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.001624 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.002357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.002657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.002937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.003633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.006032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.032953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1294/2563] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:02 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:03.037046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09133105-7bcd-409a-8222-6923a78009db None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.047655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09133105-7bcd-409a-8222-6923a78009db None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.105428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.106132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.106420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.106700 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.107390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.109302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.128753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1270/2564] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:03 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:03.133073 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.145133 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619b50a-301a-4f30-85ee-8e4a1ff7adad admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.150007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.150677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.150966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.151249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.151979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.153648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.164359 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1295/2565] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:03.168533 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.179740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6928ad8-7024-4a6c-825a-d9135fdc84bf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.184451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.185348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.185637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.185956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] PATH_INFO: `/v3/projects/3ba7025d45a74afdaa9f85e1b681943f/users/ef8afb132bdc4ffcbbb511c453e27b9e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.201682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba7025d45a74afdaa9f85e1b681943f, user_id=ef8afb132bdc4ffcbbb511c453e27b9e, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.204408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.217334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1271/2566] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:03 2022] PUT /identity/v3/projects/3ba7025d45a74afdaa9f85e1b681943f/users/ef8afb132bdc4ffcbbb511c453e27b9e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:03.222026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.232328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba2c33-4fd5-44e1-bdc8-ba1bf017be19 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.290101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b29677-bd8e-4f82-b0d4-cada6765eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.291032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29677-bd8e-4f82-b0d4-cada6765eed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.291268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29677-bd8e-4f82-b0d4-cada6765eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.291625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29677-bd8e-4f82-b0d4-cada6765eed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.292426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b29677-bd8e-4f82-b0d4-cada6765eed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.294106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b29677-bd8e-4f82-b0d4-cada6765eed5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.307956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1296/2567] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:03.309085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.319621 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f327867-bbad-45b8-9893-616450dffa87 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.324334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.325221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.325504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.325813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] PATH_INFO: `/v3/projects/3ba7025d45a74afdaa9f85e1b681943f/users/ef8afb132bdc4ffcbbb511c453e27b9e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.327754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ba7025d45a74afdaa9f85e1b681943f, user_id=ef8afb132bdc4ffcbbb511c453e27b9e, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.330948 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.342816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1272/2568] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:03 2022] PUT /identity/v3/projects/3ba7025d45a74afdaa9f85e1b681943f/users/ef8afb132bdc4ffcbbb511c453e27b9e/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:03.355240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.355534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.355818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.383531 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] MFA Rules not processed for user `ef8afb132bdc4ffcbbb511c453e27b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:03.383734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.384791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:03.385453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:03.432850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:03.444228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934698b-3e27-43ec-af23-614462eb272e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.447947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.448600 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.448890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.449174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.451070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:03.464804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1297/2569] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:03.476680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.477379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:03.478639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:03.492057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef3baa-55a3-4e24-a88e-3b0ffbe2f770 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.495685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.496335 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.496619 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.496901 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.506166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.507720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.517434 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1298/2570] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:25:03.520083 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.521521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.530559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1273/2571] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:25:03.671077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:03.679530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1405f-afee-4b7e-9971-cffcc878264b service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:03.683686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056f028-1ba0-4c7a-bd21-8ec47aff138c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:03.684380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f028-1ba0-4c7a-bd21-8ec47aff138c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:03.684615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f028-1ba0-4c7a-bd21-8ec47aff138c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:03.685013 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f028-1ba0-4c7a-bd21-8ec47aff138c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:03.746715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056f028-1ba0-4c7a-bd21-8ec47aff138c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:03.748211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056f028-1ba0-4c7a-bd21-8ec47aff138c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:03.757351 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1299/2572] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:05.624579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:05.626036 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:05.690717 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad778dff-d1e7-4b44-8032-cf8a87894a50 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:05.695106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:05.695971 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:05.696254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:05.696531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] PATH_INFO: `/v3/users/fe591c733e224cccbaac77e5050b0851` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:05.697697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe591c733e224cccbaac77e5050b0851)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:05.699623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:05.744022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe591c733e224cccbaac77e5050b0851'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:05.744668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe591c733e224cccbaac77e5050b0851'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:05.751841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe591c733e224cccbaac77e5050b0851'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:05.752453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe591c733e224cccbaac77e5050b0851'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:05.760300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe591c733e224cccbaac77e5050b0851'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:05.777499 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1274/2573] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:05 2022] DELETE /identity/v3/users/fe591c733e224cccbaac77e5050b0851 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:05.782028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:05.782542 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:05.782655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:05.799600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:05.800810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:05.858928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1300/2574] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:05.868493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:05.870017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:05.915809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:05.930829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6eb62-0049-4d10-aa61-b3b2f15938bc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:05.934437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:05.935074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:05.935387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:05.935661 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:05.945026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:05.946420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:05.956343 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1275/2575] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:06.234027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:06.236010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:06.251580 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723b9da-c5ce-4b1f-a88a-a92fedf3520c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:06.256212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:06.257108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:06.257398 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:06.258449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] PATH_INFO: `/v3/projects/c3a2831b884d4348a5bded6fae5d5796` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:06.259986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a2831b884d4348a5bded6fae5d5796)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:06.261989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:06.318858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a2831b884d4348a5bded6fae5d5796', 'request_id': 'req-86f0d003-e342-4a13-a7b7-cdf9dc613184'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.326191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a2831b884d4348a5bded6fae5d5796', 'request_id': 'req-86f0d003-e342-4a13-a7b7-cdf9dc613184'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.332506 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a2831b884d4348a5bded6fae5d5796', 'request_id': 'req-86f0d003-e342-4a13-a7b7-cdf9dc613184'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.337156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:06.344831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] The token cache is being invalidate because project c3a2831b884d4348a5bded6fae5d5796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:06.345417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.346482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.348277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1301/2576] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:06 2022] DELETE /identity/v3/projects/c3a2831b884d4348a5bded6fae5d5796 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:06.352228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:06.362718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbefb0-a2ef-4149-b070-e8dbed3a102e service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:06.418646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:06.419515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:06.419804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:06.420085 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] PATH_INFO: `/v3/users/3705454e8fd74010b7d3633e98a145c8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:06.421267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3705454e8fd74010b7d3633e98a145c8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:06.423768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:06.468698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3705454e8fd74010b7d3633e98a145c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.469344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3705454e8fd74010b7d3633e98a145c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.477031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3705454e8fd74010b7d3633e98a145c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.477613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3705454e8fd74010b7d3633e98a145c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.489391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3705454e8fd74010b7d3633e98a145c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.506819 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1276/2577] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:06 2022] DELETE /identity/v3/users/3705454e8fd74010b7d3633e98a145c8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:06.777984 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:06.779457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:06.792572 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0d003-e342-4a13-a7b7-cdf9dc613184 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:06.850944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:06.851811 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:06.852096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:06.852373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] PATH_INFO: `/v3/projects/ef040ca2d7674e25a5a9e047821b8ff3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:06.853552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef040ca2d7674e25a5a9e047821b8ff3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:06.855623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:06.900935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef040ca2d7674e25a5a9e047821b8ff3', 'request_id': 'req-ceba760b-e822-4d6f-a9f0-dbe9878916fd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.908163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef040ca2d7674e25a5a9e047821b8ff3', 'request_id': 'req-ceba760b-e822-4d6f-a9f0-dbe9878916fd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.914190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef040ca2d7674e25a5a9e047821b8ff3', 'request_id': 'req-ceba760b-e822-4d6f-a9f0-dbe9878916fd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.925655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] The token cache is being invalidate because project ef040ca2d7674e25a5a9e047821b8ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:06.926038 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.926670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:06.928188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1302/2578] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:06 2022] DELETE /identity/v3/projects/ef040ca2d7674e25a5a9e047821b8ff3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:06.932255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:06.933704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:06.947039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a114ca0-68b5-438f-85ac-d851bad28e83 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.002570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.003444 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.003729 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.004012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] PATH_INFO: `/v3/users/d5fa7720f3074fdaa0ac5d9b7c469892` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.005187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5fa7720f3074fdaa0ac5d9b7c469892)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.007142 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.009400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:07.054688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5fa7720f3074fdaa0ac5d9b7c469892'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.055287 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5fa7720f3074fdaa0ac5d9b7c469892'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.062116 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5fa7720f3074fdaa0ac5d9b7c469892'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.062741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5fa7720f3074fdaa0ac5d9b7c469892'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.070940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b7b5f1e-da99-4743-a028-42c492ad1cf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5fa7720f3074fdaa0ac5d9b7c469892'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.089707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1277/2579] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:06 2022] DELETE /identity/v3/users/d5fa7720f3074fdaa0ac5d9b7c469892 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:07.356608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:07.358187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:07.378249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceba760b-e822-4d6f-a9f0-dbe9878916fd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.435714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.436577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.436865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.437147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] PATH_INFO: `/v3/projects/e4455d03c1f6414f963c4473c374149c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.438333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4455d03c1f6414f963c4473c374149c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.440786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.471208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:07.487037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4455d03c1f6414f963c4473c374149c', 'request_id': 'req-9e112b7d-2462-4a1b-9810-b6404de9c1b6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.494285 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4455d03c1f6414f963c4473c374149c', 'request_id': 'req-9e112b7d-2462-4a1b-9810-b6404de9c1b6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.500318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4455d03c1f6414f963c4473c374149c', 'request_id': 'req-9e112b7d-2462-4a1b-9810-b6404de9c1b6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.510708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] The token cache is being invalidate because project e4455d03c1f6414f963c4473c374149c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:07.510999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.511909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9e112b7d-2462-4a1b-9810-b6404de9c1b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:07.514555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1303/2580] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:07 2022] DELETE /identity/v3/projects/e4455d03c1f6414f963c4473c374149c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:07.521007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbe37b-4473-4142-be86-b3a9c5069e8a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.521299 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbe37b-4473-4142-be86-b3a9c5069e8a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.521578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbe37b-4473-4142-be86-b3a9c5069e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.539402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-72fbe37b-4473-4142-be86-b3a9c5069e8a None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:07.540584 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-72fbe37b-4473-4142-be86-b3a9c5069e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:07.597336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1278/2581] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:07.605545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8644d23-96ce-4d6f-a4f2-bb68f9299271 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.605883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8644d23-96ce-4d6f-a4f2-bb68f9299271 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.606167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8644d23-96ce-4d6f-a4f2-bb68f9299271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.623835 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b8644d23-96ce-4d6f-a4f2-bb68f9299271 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:07.625028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8644d23-96ce-4d6f-a4f2-bb68f9299271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:07.632996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1304/2582] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:07.637061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbe37b-4473-4142-be86-b3a9c5069e8a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:07.647073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbe37b-4473-4142-be86-b3a9c5069e8a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.651738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.652394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.652787 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.653075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.653836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:07.654151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.655775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.667983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1279/2583] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:25:07.672220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8644d23-96ce-4d6f-a4f2-bb68f9299271 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:07.681917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8644d23-96ce-4d6f-a4f2-bb68f9299271 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.686673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.687381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.687667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.687944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.689230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.691167 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.715900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1305/2584] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:07 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:07.720196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:07.730419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868421cc-3889-40f2-9ddf-f73a2dadf4cc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.786275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.786940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.787226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.787507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.788198 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.790148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.804799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:07.811281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1280/2585] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:07 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:07.815548 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:07.826139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe52f72-1530-45c8-afff-4fc47eec85ae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.831264 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.831966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.832435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.832555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.833227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.834867 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.845315 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1306/2586] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:07.851072 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:07.861403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddea00-e234-473a-a744-12e6fd8ffb27 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.866084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92f631-92e3-45db-997d-c0845f142be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.866740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92f631-92e3-45db-997d-c0845f142be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.867027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92f631-92e3-45db-997d-c0845f142be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.867303 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92f631-92e3-45db-997d-c0845f142be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.868013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92f631-92e3-45db-997d-c0845f142be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.869595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92f631-92e3-45db-997d-c0845f142be0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.880070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1281/2587] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:07.884100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:07.894189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fefcd-df5b-452d-a101-5f376ba62eda admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:07.898826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:07.899705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.899994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.900272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] PATH_INFO: `/v3/projects/6ca307ec84b345d1a5b6bc3842254206/users/c6305bead0094efaa2834cd20d860402/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.915283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca307ec84b345d1a5b6bc3842254206, user_id=c6305bead0094efaa2834cd20d860402, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:07.917968 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:07.930779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1307/2588] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:07 2022] PUT /identity/v3/projects/6ca307ec84b345d1a5b6bc3842254206/users/c6305bead0094efaa2834cd20d860402/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:07.941857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a592e87-7824-4e08-9d99-44192403705e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:07.942348 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a592e87-7824-4e08-9d99-44192403705e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:07.942467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a592e87-7824-4e08-9d99-44192403705e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:07.968146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0a592e87-7824-4e08-9d99-44192403705e None None] MFA Rules not processed for user `c6305bead0094efaa2834cd20d860402`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:07.969337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0a592e87-7824-4e08-9d99-44192403705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:08.031962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1282/2589] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:08.036920 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.098950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cc695-b9d6-41b7-988e-34b1a640721b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.103814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.104406 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.104690 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.104970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.105659 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.107670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.132317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1308/2590] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:08 2022] POST /identity/v3/projects => generated 393 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:08.136452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a592e87-7824-4e08-9d99-44192403705e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.147576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a592e87-7824-4e08-9d99-44192403705e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.202442 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.203219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.203501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.203778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.204486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.206427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.226831 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1283/2591] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:08 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:08.230742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.241444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190e2cb-b981-414d-966a-f06f7ce0451a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.246036 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.246706 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.246971 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.247248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.247972 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.249698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.260061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1309/2592] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:08.263964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.274389 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0b2cf-6b7b-45ea-be1a-4953a93d9bd2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.278941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.279810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.280095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.280375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] PATH_INFO: `/v3/projects/cb132c71d9014d469c3e0f9b8c4853e3/users/56284ade18ef4770bef89ed61b0af461/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.296595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb132c71d9014d469c3e0f9b8c4853e3, user_id=56284ade18ef4770bef89ed61b0af461, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.300294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.313357 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1284/2593] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:08 2022] PUT /identity/v3/projects/cb132c71d9014d469c3e0f9b8c4853e3/users/56284ade18ef4770bef89ed61b0af461/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:08.317943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.328233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1668e3-a2d0-448f-a62e-25d447bfde6a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.352505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:08.388727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a99b0-7c93-49dc-a406-e80ec68ce625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.389328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a99b0-7c93-49dc-a406-e80ec68ce625 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.389638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a99b0-7c93-49dc-a406-e80ec68ce625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.389948 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a99b0-7c93-49dc-a406-e80ec68ce625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.390669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a99b0-7c93-49dc-a406-e80ec68ce625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.392331 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a99b0-7c93-49dc-a406-e80ec68ce625 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.402461 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1310/2594] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:08.406903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.417150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212d839-968c-42ab-b88f-b051e7a5e4a3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.421794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.423201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.423493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.423772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] PATH_INFO: `/v3/projects/cb132c71d9014d469c3e0f9b8c4853e3/users/56284ade18ef4770bef89ed61b0af461/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.425746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb132c71d9014d469c3e0f9b8c4853e3, user_id=56284ade18ef4770bef89ed61b0af461, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.428365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.438577 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1285/2595] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:08 2022] PUT /identity/v3/projects/cb132c71d9014d469c3e0f9b8c4853e3/users/56284ade18ef4770bef89ed61b0af461/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:08.449298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.449618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.449931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.476623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] MFA Rules not processed for user `56284ade18ef4770bef89ed61b0af461`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:08.477895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:08.543069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1311/2596] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:08.554111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.555848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:08.600216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:08.627399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ece97-4dad-4874-9513-da1033947acd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.631041 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.631682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.631970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.632263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.641239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.643177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.652557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1286/2597] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:08.699226 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:08.700693 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:08.712671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504e49a-221e-4fe8-a8b9-bdc66b137b4f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:08.716325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ee4ed-bf2e-41f2-90ac-27c7f729b358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:08.716988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ee4ed-bf2e-41f2-90ac-27c7f729b358 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:08.717278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ee4ed-bf2e-41f2-90ac-27c7f729b358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:08.717560 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ee4ed-bf2e-41f2-90ac-27c7f729b358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:08.774253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ee4ed-bf2e-41f2-90ac-27c7f729b358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:08.775707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ee4ed-bf2e-41f2-90ac-27c7f729b358 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:08.785219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1312/2598] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:12.051090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:12.116133 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b9257-7e23-4cd3-98d0-e3fcdff385bc service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:12.120888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:12.121845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:12.122131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:12.122409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] PATH_INFO: `/v3/users/c6305bead0094efaa2834cd20d860402` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:12.123657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6305bead0094efaa2834cd20d860402)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:12.126145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:12.175813 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6305bead0094efaa2834cd20d860402'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.176475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6305bead0094efaa2834cd20d860402'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.183302 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6305bead0094efaa2834cd20d860402'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.184004 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6305bead0094efaa2834cd20d860402'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.192673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6305bead0094efaa2834cd20d860402'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.210279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1287/2599] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:12 2022] DELETE /identity/v3/users/c6305bead0094efaa2834cd20d860402 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:12.214793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:12.215284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:12.215449 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:12.233015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:12.234483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:12.275265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:12.293426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1313/2600] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:12.303555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:12.304929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:12.362827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e990cb1-8fa6-4fb0-a04b-1304cec19fef admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:12.366500 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:12.367156 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:12.367403 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:12.367683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:12.373690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:12.378776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:12.381139 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:12.391021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1288/2601] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:12.663738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:12.665203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:12.677236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486399da-309e-48d3-bf4d-d0cabe24baf0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:12.681809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:12.682676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:12.682966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:12.683253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] PATH_INFO: `/v3/projects/6ca307ec84b345d1a5b6bc3842254206` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:12.684520 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca307ec84b345d1a5b6bc3842254206)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:12.686482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:12.735893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca307ec84b345d1a5b6bc3842254206', 'request_id': 'req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.743329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca307ec84b345d1a5b6bc3842254206', 'request_id': 'req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.749470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca307ec84b345d1a5b6bc3842254206', 'request_id': 'req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.759813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] The token cache is being invalidate because project 6ca307ec84b345d1a5b6bc3842254206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:12.760100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.760733 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.762309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1314/2602] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:12 2022] DELETE /identity/v3/projects/6ca307ec84b345d1a5b6bc3842254206 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:12.766748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:12.776328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ea9dd-4956-4b4d-9b0c-ea3cf6657d8f service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:12.832263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:12.833137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:12.833425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:12.833708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] PATH_INFO: `/v3/users/56284ade18ef4770bef89ed61b0af461` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:12.834913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56284ade18ef4770bef89ed61b0af461)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:12.836815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:12.881135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56284ade18ef4770bef89ed61b0af461'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.881810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56284ade18ef4770bef89ed61b0af461'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.888541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56284ade18ef4770bef89ed61b0af461'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.889166 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56284ade18ef4770bef89ed61b0af461'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.897022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56284ade18ef4770bef89ed61b0af461'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:12.914224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1289/2603] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:12 2022] DELETE /identity/v3/users/56284ade18ef4770bef89ed61b0af461 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:13.122400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.123804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.139076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bf5f7-d719-4bd5-8e0f-c4a73307d214 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.203120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.205546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.206055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.206494 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] PATH_INFO: `/v3/users/0b5b4df3ceea413cab57f196e5df36a0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.210097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5b4df3ceea413cab57f196e5df36a0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.214510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.232937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.234924 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.254249 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81cde9-16b6-4c38-b715-3a73576ae491 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.260655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.261527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.261851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.262132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] PATH_INFO: `/v3/projects/cb132c71d9014d469c3e0f9b8c4853e3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.263422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb132c71d9014d469c3e0f9b8c4853e3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.266191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.286946 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5b4df3ceea413cab57f196e5df36a0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.287786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5b4df3ceea413cab57f196e5df36a0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.288551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5b4df3ceea413cab57f196e5df36a0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.297302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5b4df3ceea413cab57f196e5df36a0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.307328 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-77982354-60bf-423b-9b87-84581426eb65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5b4df3ceea413cab57f196e5df36a0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.315870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:13.319023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb132c71d9014d469c3e0f9b8c4853e3', 'request_id': 'req-4f3f1f54-671b-43cb-b417-ab94af700e25'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.327374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb132c71d9014d469c3e0f9b8c4853e3', 'request_id': 'req-4f3f1f54-671b-43cb-b417-ab94af700e25'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.331942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1315/2604] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:13 2022] DELETE /identity/v3/users/0b5b4df3ceea413cab57f196e5df36a0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:13.336614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.337052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.337464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.337865 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb132c71d9014d469c3e0f9b8c4853e3', 'request_id': 'req-4f3f1f54-671b-43cb-b417-ab94af700e25'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.342612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:13.351381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] The token cache is being invalidate because project cb132c71d9014d469c3e0f9b8c4853e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:13.351667 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.352781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f3f1f54-671b-43cb-b417-ab94af700e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.357447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1290/2605] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:13 2022] DELETE /identity/v3/projects/cb132c71d9014d469c3e0f9b8c4853e3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:13.358395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:13.358862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502860a-ee1e-4808-9fd0-77f04b41ce03 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.359149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502860a-ee1e-4808-9fd0-77f04b41ce03 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.359427 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502860a-ee1e-4808-9fd0-77f04b41ce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.359830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.378701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d502860a-ee1e-4808-9fd0-77f04b41ce03 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:13.379879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d502860a-ee1e-4808-9fd0-77f04b41ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.420047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1316/2606] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:13.430786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.432106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.440102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1291/2607] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:13.448630 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c44ec-372d-4cec-ae74-1e83c9139dff None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.448922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c44ec-372d-4cec-ae74-1e83c9139dff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.449206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c44ec-372d-4cec-ae74-1e83c9139dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.474326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4f3c44ec-372d-4cec-ae74-1e83c9139dff None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:13.475534 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f3c44ec-372d-4cec-ae74-1e83c9139dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.483474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1292/2608] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:13.488115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c44ec-372d-4cec-ae74-1e83c9139dff None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.493942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9d719-487c-4fd3-b84c-428480386770 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.497226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c44ec-372d-4cec-ae74-1e83c9139dff None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.497667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.498966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.499278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.499559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.502013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.502673 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.502953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.503223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.503929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:13.504231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.505841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.508803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.510278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.521316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1293/2609] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:25:13.521490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1317/2610] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:25:13.522390 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.532574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86243c67-b807-4012-99e8-8132a6da306a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.537388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.538136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.538492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.538849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.539602 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.541558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.570297 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1294/2611] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:13 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:13.574976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.587278 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c12998-f27c-4ffe-b720-c6a83a59b2a1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.651642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.652460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.652752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.655818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.655818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.655818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.683915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1318/2612] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:13 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:13.684977 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.695517 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d9f8c-e825-4550-b5a6-823cbdc5e8ec admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.700944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.701610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.701922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.702194 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.702909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.704549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.715034 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1295/2613] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:13.719108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.730009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea68153-5d83-4d5e-a6f8-fe443ee710d6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.734930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.736183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.736471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.736746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.737469 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.739140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.749559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1319/2614] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:13.753654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.764236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15004ebd-a1ed-4b50-ba57-37bba03d845b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.771817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768fb7a-371a-4189-ace7-6534bf268bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.772712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb7a-371a-4189-ace7-6534bf268bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.773003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb7a-371a-4189-ace7-6534bf268bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.773289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768fb7a-371a-4189-ace7-6534bf268bd4 admin admin] PATH_INFO: `/v3/projects/b9998bfb7bad49c3824ed5cf7cb5ead7/users/3121596745114fc29daedbfbd2cf783a/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.789488 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.791318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.792062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768fb7a-371a-4189-ace7-6534bf268bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9998bfb7bad49c3824ed5cf7cb5ead7, user_id=3121596745114fc29daedbfbd2cf783a, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.795061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768fb7a-371a-4189-ace7-6534bf268bd4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.812720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1296/2615] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:13 2022] PUT /identity/v3/projects/b9998bfb7bad49c3824ed5cf7cb5ead7/users/3121596745114fc29daedbfbd2cf783a/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:13.816970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dc302-2b8a-4291-ad6b-139c377240bb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.823796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9084fe-7c65-410a-8ce9-612a4f9b167b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.824088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9084fe-7c65-410a-8ce9-612a4f9b167b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.824429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9084fe-7c65-410a-8ce9-612a4f9b167b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.828050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:13.854512 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1c9084fe-7c65-410a-8ce9-612a4f9b167b None None] MFA Rules not processed for user `3121596745114fc29daedbfbd2cf783a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:13.855855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1c9084fe-7c65-410a-8ce9-612a4f9b167b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:13.883565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:13.884431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:13.884714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:13.884991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] PATH_INFO: `/v3/projects/03b94ea091b94ce99cd7495b6682f0b1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:13.886194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b94ea091b94ce99cd7495b6682f0b1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:13.888097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:13.940453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1297/2616] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:13.944791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9084fe-7c65-410a-8ce9-612a4f9b167b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.955804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b94ea091b94ce99cd7495b6682f0b1', 'request_id': 'req-5ae2802d-d2a9-4efa-a826-65496ad87758'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.958658 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9084fe-7c65-410a-8ce9-612a4f9b167b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:13.964381 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:13.965289 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b94ea091b94ce99cd7495b6682f0b1', 'request_id': 'req-5ae2802d-d2a9-4efa-a826-65496ad87758'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.975141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b94ea091b94ce99cd7495b6682f0b1', 'request_id': 'req-5ae2802d-d2a9-4efa-a826-65496ad87758'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.986792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] The token cache is being invalidate because project 03b94ea091b94ce99cd7495b6682f0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:13.987173 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.987965 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:13.989599 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1320/2617] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:13 2022] DELETE /identity/v3/projects/03b94ea091b94ce99cd7495b6682f0b1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:13.993633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:13.995031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.023861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.024558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.024830 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.025163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.026105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.027990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.055707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1298/2618] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:13 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:14.059845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.061852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.062518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2802d-d2a9-4efa-a826-65496ad87758 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.116225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:14.140163 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.141771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.142120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.143161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] PATH_INFO: `/v3/users/ef8afb132bdc4ffcbbb511c453e27b9e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.143646 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef8afb132bdc4ffcbbb511c453e27b9e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.145597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a7325-ae31-45b6-a735-8e9c01ca1ce0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.145766 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.151479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.152981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.153269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.153543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.154277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.156254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.185210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1299/2619] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:14 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:14.189317 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.199338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f952377-d3e0-44a2-904b-f319bfb70049 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.199510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef8afb132bdc4ffcbbb511c453e27b9e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.200362 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef8afb132bdc4ffcbbb511c453e27b9e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.201048 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef8afb132bdc4ffcbbb511c453e27b9e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.206410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.207225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.207564 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.207789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.208613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.210386 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.211105 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef8afb132bdc4ffcbbb511c453e27b9e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.219630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef8afb132bdc4ffcbbb511c453e27b9e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.226625 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1300/2620] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:14.227542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.229441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.241422 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1321/2621] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:13 2022] DELETE /identity/v3/users/ef8afb132bdc4ffcbbb511c453e27b9e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:14.246759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cf42c-26db-45bb-aec6-fe6f9d4f021d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.319815 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.320797 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.321068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.321364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] PATH_INFO: `/v3/projects/a25437137b884e7d983fe4de9ba9cff1/users/6382db5b5e174828ba38964e6b30a78f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.338213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25437137b884e7d983fe4de9ba9cff1, user_id=6382db5b5e174828ba38964e6b30a78f, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.340921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.344000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:14.363162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1301/2622] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:14 2022] PUT /identity/v3/projects/a25437137b884e7d983fe4de9ba9cff1/users/6382db5b5e174828ba38964e6b30a78f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:14.367377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.378033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899783aa-fdc5-4ebb-8fe0-a24e331cac99 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.453240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabb9c8-f66f-49ca-a9f4-704e1b356cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.454680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabb9c8-f66f-49ca-a9f4-704e1b356cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.454969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabb9c8-f66f-49ca-a9f4-704e1b356cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.455250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabb9c8-f66f-49ca-a9f4-704e1b356cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.456092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabb9c8-f66f-49ca-a9f4-704e1b356cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.457593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabb9c8-f66f-49ca-a9f4-704e1b356cd4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.468265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1322/2623] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:14.472503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.482957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e5073-cb72-40c9-afb3-e169ecee93a7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.487758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.488647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.488930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.489206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] PATH_INFO: `/v3/projects/a25437137b884e7d983fe4de9ba9cff1/users/6382db5b5e174828ba38964e6b30a78f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.491240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25437137b884e7d983fe4de9ba9cff1, user_id=6382db5b5e174828ba38964e6b30a78f, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.493925 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.505026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1302/2624] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:14 2022] PUT /identity/v3/projects/a25437137b884e7d983fe4de9ba9cff1/users/6382db5b5e174828ba38964e6b30a78f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:14.515540 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.515949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.516307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.533265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.535082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.549121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] MFA Rules not processed for user `6382db5b5e174828ba38964e6b30a78f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:14.550377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.602435 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:14.612448 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98231510-2d50-467f-a476-a569b1918a2e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.621110 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.622205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.622567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.622916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] PATH_INFO: `/v3/projects/3ba7025d45a74afdaa9f85e1b681943f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.623565 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1323/2625] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:14.628100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ba7025d45a74afdaa9f85e1b681943f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.630110 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.635623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.637682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.680319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ba7025d45a74afdaa9f85e1b681943f', 'request_id': 'req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.688981 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ba7025d45a74afdaa9f85e1b681943f', 'request_id': 'req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.695410 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ba7025d45a74afdaa9f85e1b681943f', 'request_id': 'req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.697458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454ea1f-1c3c-4080-9a89-8d01112e39c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.706458 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] The token cache is being invalidate because project 3ba7025d45a74afdaa9f85e1b681943f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:14.706663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.707313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ec6f465f-3c63-4ab3-bdce-de01b7b768ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:14.708894 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1303/2626] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:14 2022] DELETE /identity/v3/projects/3ba7025d45a74afdaa9f85e1b681943f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:14.713622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568279-bc5b-4f9a-b588-099ca26d7227 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.713943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568279-bc5b-4f9a-b588-099ca26d7227 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.714220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568279-bc5b-4f9a-b588-099ca26d7227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.732134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-13568279-bc5b-4f9a-b588-099ca26d7227 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:14.733369 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-13568279-bc5b-4f9a-b588-099ca26d7227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.750137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.751410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.751705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.751987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.754267 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.792613 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1304/2627] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:14.800899 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6b0df-f96c-4d2a-8337-bdc7d8724a6f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.801187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6b0df-f96c-4d2a-8337-bdc7d8724a6f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.801468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6b0df-f96c-4d2a-8337-bdc7d8724a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.819467 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.819828 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bfe6b0df-f96c-4d2a-8337-bdc7d8724a6f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:14.820950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.821067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bfe6b0df-f96c-4d2a-8337-bdc7d8724a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.828928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1305/2628] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:14.831390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1324/2629] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:14.832868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6b0df-f96c-4d2a-8337-bdc7d8724a6f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.842059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6b0df-f96c-4d2a-8337-bdc7d8724a6f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.847447 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.848098 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.848380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.848656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.849371 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:14.849678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.851320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.853027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:14.869217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1306/2630] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:25:14.870107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.882148 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8019c-81c5-41d7-b412-3fcfe63b3c97 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.886666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.887323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.887608 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.887914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.888602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.890535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.917912 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1325/2631] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:14 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:14.921701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.932531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0823bf-8092-44ac-9b5b-50b2baed2625 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.933370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:14.935096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:14.970880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:14.991543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.992221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.992506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.992796 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:14.993492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:14.995037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d070b82-622e-4b76-a4ba-187e3e031b91 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:14.995457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:14.998709 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:14.999347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:14.999634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:14.999910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.001807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:15.015766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1307/2632] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:14 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:15.020037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.029491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755323a-1236-4e90-9894-ebfe640a883b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.034646 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.035286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.035571 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.035855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.036561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.038176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.051647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1308/2633] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:15.052702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.062175 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c8f43-76af-4099-b967-70cad9c1bed1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.064497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.065959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.067331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3620a4-8c36-4b1b-920f-e87f1f9a9326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.067979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3620a4-8c36-4b1b-920f-e87f1f9a9326 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.068265 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3620a4-8c36-4b1b-920f-e87f1f9a9326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.068546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3620a4-8c36-4b1b-920f-e87f1f9a9326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.069255 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3620a4-8c36-4b1b-920f-e87f1f9a9326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.070866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3620a4-8c36-4b1b-920f-e87f1f9a9326 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.075245 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1326/2634] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:25:15.083147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1309/2635] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:15.087265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.098614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524393a1-80dc-4732-9bb3-096d1ac15570 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.103732 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.104618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.104906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.105178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] PATH_INFO: `/v3/projects/194b07895534427994b6200c090b6c06/users/20086b55cedc49f3819b9e0a50a0f914/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.120410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] RBAC: Authorizing `identity:create_grant(project_id=194b07895534427994b6200c090b6c06, user_id=20086b55cedc49f3819b9e0a50a0f914, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.123189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.140072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1327/2636] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:15 2022] PUT /identity/v3/projects/194b07895534427994b6200c090b6c06/users/20086b55cedc49f3819b9e0a50a0f914/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:15.151598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7800d4-cd61-416e-b4ea-6de002871fcc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.151890 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7800d4-cd61-416e-b4ea-6de002871fcc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.152288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7800d4-cd61-416e-b4ea-6de002871fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.178976 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5f7800d4-cd61-416e-b4ea-6de002871fcc None None] MFA Rules not processed for user `20086b55cedc49f3819b9e0a50a0f914`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:15.180180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5f7800d4-cd61-416e-b4ea-6de002871fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:15.253316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1310/2637] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:15.257797 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.325959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09263aa-7fca-406c-a51b-2e0cf2f9096d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.329350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.330161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.330689 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.331097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.332027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.334258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.366889 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1328/2638] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:15 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:15.368404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7800d4-cd61-416e-b4ea-6de002871fcc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.386052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7800d4-cd61-416e-b4ea-6de002871fcc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.435511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:15.446520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.447191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.447480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.447762 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.448478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.450420 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.470490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1311/2639] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:15 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:15.474780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.484919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30c0b8f-e683-4c7a-941f-59d93cb10166 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.489666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.490450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.490739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.491053 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.491962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.493630 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.504270 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1329/2640] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:15.508346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.518731 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80287262-1994-4f8b-aa44-ed296ef2bb99 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.523268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.524138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.524422 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.524702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] PATH_INFO: `/v3/projects/da7d1da5f0bc411f88ace76b3c73f060/users/0fdfd3e5cb7d44ef913a1a379707ac32/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.539491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7d1da5f0bc411f88ace76b3c73f060, user_id=0fdfd3e5cb7d44ef913a1a379707ac32, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.542171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.555017 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1312/2641] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:15 2022] PUT /identity/v3/projects/da7d1da5f0bc411f88ace76b3c73f060/users/0fdfd3e5cb7d44ef913a1a379707ac32/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:15.558858 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.568991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf23f73-5f5a-42ba-b99b-d0f28f76f368 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.601937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:15.625275 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e89d3-5fab-4bd8-aad0-d2a8b1680e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.625975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e89d3-5fab-4bd8-aad0-d2a8b1680e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.626259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e89d3-5fab-4bd8-aad0-d2a8b1680e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.626536 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e89d3-5fab-4bd8-aad0-d2a8b1680e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.627248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e89d3-5fab-4bd8-aad0-d2a8b1680e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.628846 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e89d3-5fab-4bd8-aad0-d2a8b1680e0b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.638358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1330/2642] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:15.643039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.653119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8b431-03a3-47f3-ac88-3f72941d02b2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.658281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.659149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.659434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.659713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] PATH_INFO: `/v3/projects/da7d1da5f0bc411f88ace76b3c73f060/users/0fdfd3e5cb7d44ef913a1a379707ac32/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.661654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7d1da5f0bc411f88ace76b3c73f060, user_id=0fdfd3e5cb7d44ef913a1a379707ac32, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.664307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.675187 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1313/2643] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:15 2022] PUT /identity/v3/projects/da7d1da5f0bc411f88ace76b3c73f060/users/0fdfd3e5cb7d44ef913a1a379707ac32/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:15.684996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.685295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.685578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.716501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] MFA Rules not processed for user `0fdfd3e5cb7d44ef913a1a379707ac32`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:15.717743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:15.789431 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1331/2644] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:15.800972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:15.856025 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f976932-2f83-4971-8ff0-d28af62f7a9e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:15.859664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1979a63-7dcf-4e18-b1fb-8463d1192f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:15.860832 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1979a63-7dcf-4e18-b1fb-8463d1192f4b service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:15.861118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1979a63-7dcf-4e18-b1fb-8463d1192f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:15.861395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1979a63-7dcf-4e18-b1fb-8463d1192f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:15.916256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1979a63-7dcf-4e18-b1fb-8463d1192f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:15.917685 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1979a63-7dcf-4e18-b1fb-8463d1192f4b service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:15.927039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1314/2645] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:17.652344 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:17.653871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:17.722616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c8bf4-902d-4e20-9a64-62916c9c9741 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:17.727205 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:17.728077 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:17.728357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:17.728631 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] PATH_INFO: `/v3/users/3121596745114fc29daedbfbd2cf783a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:17.729820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3121596745114fc29daedbfbd2cf783a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:17.731735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:17.776831 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3121596745114fc29daedbfbd2cf783a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:17.777541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3121596745114fc29daedbfbd2cf783a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:17.778258 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3121596745114fc29daedbfbd2cf783a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:17.786146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3121596745114fc29daedbfbd2cf783a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:17.791395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:17.793484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3121596745114fc29daedbfbd2cf783a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:17.813082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1332/2646] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:17 2022] DELETE /identity/v3/users/3121596745114fc29daedbfbd2cf783a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:17.817643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:17.818174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:17.818293 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:17.835827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:17.837014 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:17.886233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:17.896960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1315/2647] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:17.907457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:17.908857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:17.966216 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd0a36-eabe-46c4-a4aa-32fdb10f6516 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:17.969911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:17.970555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:17.970837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:17.971113 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:17.980178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:17.981607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:17.991692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1333/2648] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:18.257984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:18.259513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:18.275474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c915f-4e8e-440b-acb0-fee22d027dd0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:18.280233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:18.281002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.281286 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.281650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] PATH_INFO: `/v3/projects/b9998bfb7bad49c3824ed5cf7cb5ead7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.282845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9998bfb7bad49c3824ed5cf7cb5ead7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:18.284751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:18.330954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9998bfb7bad49c3824ed5cf7cb5ead7', 'request_id': 'req-4f25be30-667a-4a5d-805d-bb081156597a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.338089 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9998bfb7bad49c3824ed5cf7cb5ead7', 'request_id': 'req-4f25be30-667a-4a5d-805d-bb081156597a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.344227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9998bfb7bad49c3824ed5cf7cb5ead7', 'request_id': 'req-4f25be30-667a-4a5d-805d-bb081156597a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.354827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] The token cache is being invalidate because project b9998bfb7bad49c3824ed5cf7cb5ead7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:18.355114 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.355738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.357239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1316/2649] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:18 2022] DELETE /identity/v3/projects/b9998bfb7bad49c3824ed5cf7cb5ead7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:18.361218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:18.370660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3839d9f-9a19-4bc3-aa20-12ec1c0fcab4 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:18.426480 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:18.427284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.428050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.428050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] PATH_INFO: `/v3/users/6382db5b5e174828ba38964e6b30a78f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.428999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6382db5b5e174828ba38964e6b30a78f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:18.430937 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:18.448551 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:18.450071 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:18.465532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25be30-667a-4a5d-805d-bb081156597a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:18.471125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:18.472507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.472898 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.473207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] PATH_INFO: `/v3/users/20086b55cedc49f3819b9e0a50a0f914` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.474709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20086b55cedc49f3819b9e0a50a0f914)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:18.476645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:18.488906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6382db5b5e174828ba38964e6b30a78f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.489555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6382db5b5e174828ba38964e6b30a78f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.490194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6382db5b5e174828ba38964e6b30a78f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.498829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6382db5b5e174828ba38964e6b30a78f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.507853 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-402aba7c-f71f-4d5b-9dba-2cb005b21e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6382db5b5e174828ba38964e6b30a78f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.521736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:18.526563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1334/2650] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:18 2022] DELETE /identity/v3/users/6382db5b5e174828ba38964e6b30a78f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:18.531817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20086b55cedc49f3819b9e0a50a0f914'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.532520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20086b55cedc49f3819b9e0a50a0f914'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.539730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20086b55cedc49f3819b9e0a50a0f914'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.540397 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20086b55cedc49f3819b9e0a50a0f914'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.548983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20086b55cedc49f3819b9e0a50a0f914'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.568062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1317/2651] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:18 2022] DELETE /identity/v3/users/20086b55cedc49f3819b9e0a50a0f914 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:18.572522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.572818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.573098 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.590055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:18.591394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:18.654779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1335/2652] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:18.664934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:18.666347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:18.738882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e62ae-77b6-4f17-836f-c114b5908732 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:18.742622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c8caf-60b1-481e-b97b-34f2afede144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:18.743272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c8caf-60b1-481e-b97b-34f2afede144 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.743559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c8caf-60b1-481e-b97b-34f2afede144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.743841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c8caf-60b1-481e-b97b-34f2afede144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.751823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:18.755071 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c8caf-60b1-481e-b97b-34f2afede144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:18.756705 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c8caf-60b1-481e-b97b-34f2afede144 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:18.767884 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1318/2653] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:18.840343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:18.841804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:18.854598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec9b12-780a-4b90-a66e-2f332b2e3ff9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:18.861982 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:18.863680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.863983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.864268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] PATH_INFO: `/v3/projects/a25437137b884e7d983fe4de9ba9cff1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.865568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25437137b884e7d983fe4de9ba9cff1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:18.867607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:18.919591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25437137b884e7d983fe4de9ba9cff1', 'request_id': 'req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.928155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25437137b884e7d983fe4de9ba9cff1', 'request_id': 'req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.934822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25437137b884e7d983fe4de9ba9cff1', 'request_id': 'req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.945254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] The token cache is being invalidate because project a25437137b884e7d983fe4de9ba9cff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:18.945625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.946372 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0e85896f-eacd-49f7-bd9b-9e87f55dbfaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:18.947959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1336/2654] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:18 2022] DELETE /identity/v3/projects/a25437137b884e7d983fe4de9ba9cff1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:18.955345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923351-7e9e-49ff-9559-75b47bf35710 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:18.955345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923351-7e9e-49ff-9559-75b47bf35710 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:18.955345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69923351-7e9e-49ff-9559-75b47bf35710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:18.975441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-69923351-7e9e-49ff-9559-75b47bf35710 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:18.976961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69923351-7e9e-49ff-9559-75b47bf35710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.041225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1319/2655] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:19.049502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.049836 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.050122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.068543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:19.069897 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.079187 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1337/2656] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:19.083464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923351-7e9e-49ff-9559-75b47bf35710 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.088806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.090266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.094954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69923351-7e9e-49ff-9559-75b47bf35710 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.102827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.103191 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab8532-e18d-40df-9412-0e1200303b11 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.103501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.103847 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.104057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.104782 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:19.105086 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.107267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.108486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.109281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.109553 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.110535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] PATH_INFO: `/v3/projects/194b07895534427994b6200c090b6c06` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.111677 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=194b07895534427994b6200c090b6c06)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.113586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.121938 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1320/2657] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:19.126221 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.141992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509e5b8-dd68-4971-bea8-9c2eab076b2c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.147781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.148439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.148720 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.148995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.150104 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.152010 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.164793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194b07895534427994b6200c090b6c06', 'request_id': 'req-2c883043-03ed-4c91-98ed-aa0a355a4103'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.174457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194b07895534427994b6200c090b6c06', 'request_id': 'req-2c883043-03ed-4c91-98ed-aa0a355a4103'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.181058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194b07895534427994b6200c090b6c06', 'request_id': 'req-2c883043-03ed-4c91-98ed-aa0a355a4103'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.181350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1321/2658] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:25:19.185488 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.188323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:19.192833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] The token cache is being invalidate because project 194b07895534427994b6200c090b6c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:19.193803 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.195009 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.197685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1338/2659] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:19 2022] DELETE /identity/v3/projects/194b07895534427994b6200c090b6c06 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:19.201460 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.204141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.253496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be57f9-e749-4b8c-b214-29f1f3227e63 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.258234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.258907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.259190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.259469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.260151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.262130 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.270162 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c883043-03ed-4c91-98ed-aa0a355a4103 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.275557 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.276477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.276848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.277139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] PATH_INFO: `/v3/users/0fdfd3e5cb7d44ef913a1a379707ac32` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.278388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdfd3e5cb7d44ef913a1a379707ac32)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.280326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.283337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1322/2660] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:19.287320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.288963 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.301106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b3d26-476b-4ef8-aed5-f79628a00a42 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.305945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.306752 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.306992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.307275 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.307989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.309576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.319799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1323/2661] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:19.323760 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.330492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdfd3e5cb7d44ef913a1a379707ac32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.331153 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdfd3e5cb7d44ef913a1a379707ac32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.331768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdfd3e5cb7d44ef913a1a379707ac32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.333960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ffb9f1-bfb6-43d6-a390-a483e5e23856 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.338647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.339292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.339593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.339782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdfd3e5cb7d44ef913a1a379707ac32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.339945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.340605 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.342208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.348298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f131160-590a-4dfe-a500-21a33bc1c6aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdfd3e5cb7d44ef913a1a379707ac32'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.352997 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1324/2662] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:25:19.356998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.358747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.366739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1339/2663] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:19 2022] DELETE /identity/v3/users/0fdfd3e5cb7d44ef913a1a379707ac32 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:19.371939 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcc151-d4a3-4cac-9f9b-5f94f2675a9a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.398524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:19.431747 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.432638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.432930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.433234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] PATH_INFO: `/v3/projects/6293b2fd55344b2e9fe316677d5beff4/users/382086525a1d4e39bbf01e69524cb504/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.448170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6293b2fd55344b2e9fe316677d5beff4, user_id=382086525a1d4e39bbf01e69524cb504, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.450809 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.464495 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1325/2664] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:19 2022] PUT /identity/v3/projects/6293b2fd55344b2e9fe316677d5beff4/users/382086525a1d4e39bbf01e69524cb504/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:25:19.475353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.475654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.475942 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.503611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] MFA Rules not processed for user `382086525a1d4e39bbf01e69524cb504`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:19.504817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.576662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1340/2665] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:19.587891 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.589712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.641941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.645450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.654911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:19.660913 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab3ffc-4453-48ae-8c27-0a5f9abebf2d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.665157 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabe8c6-3347-4da0-aa5c-24fba429e910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.665852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe8c6-3347-4da0-aa5c-24fba429e910 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.666137 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe8c6-3347-4da0-aa5c-24fba429e910 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.666415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe8c6-3347-4da0-aa5c-24fba429e910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.676308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabe8c6-3347-4da0-aa5c-24fba429e910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.677800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabe8c6-3347-4da0-aa5c-24fba429e910 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.686800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1326/2666] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:19.718916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f89db4-0226-46b9-9b49-611e1fcc296d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.724930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.725850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.726139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.726454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] PATH_INFO: `/v3/projects/da7d1da5f0bc411f88ace76b3c73f060` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.727692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7d1da5f0bc411f88ace76b3c73f060)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.729758 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.778120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7d1da5f0bc411f88ace76b3c73f060', 'request_id': 'req-cae24d28-a804-4676-a8b1-a6194305edbb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.786201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7d1da5f0bc411f88ace76b3c73f060', 'request_id': 'req-cae24d28-a804-4676-a8b1-a6194305edbb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.793069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7d1da5f0bc411f88ace76b3c73f060', 'request_id': 'req-cae24d28-a804-4676-a8b1-a6194305edbb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.804576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] The token cache is being invalidate because project da7d1da5f0bc411f88ace76b3c73f060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:19.804864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.805747 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cae24d28-a804-4676-a8b1-a6194305edbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:19.807622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1341/2667] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:19 2022] DELETE /identity/v3/projects/da7d1da5f0bc411f88ace76b3c73f060 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:19.813491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0f143-83d8-448d-9c17-2130f33c2764 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.813774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0f143-83d8-448d-9c17-2130f33c2764 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.814123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0f143-83d8-448d-9c17-2130f33c2764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.846126 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-23e0f143-83d8-448d-9c17-2130f33c2764 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:19.848058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-23e0f143-83d8-448d-9c17-2130f33c2764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.915253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1327/2668] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:25:19.924286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503cbf-da38-43d8-a368-976451593d55 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.924585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503cbf-da38-43d8-a368-976451593d55 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.924866 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c503cbf-da38-43d8-a368-976451593d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.945868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6c503cbf-da38-43d8-a368-976451593d55 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:19.947095 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c503cbf-da38-43d8-a368-976451593d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:19.955621 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1342/2669] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:19.959657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0f143-83d8-448d-9c17-2130f33c2764 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:19.971307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0f143-83d8-448d-9c17-2130f33c2764 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:19.976433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:19.977224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:19.977508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:19.977886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:19.978623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:19.978931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:19.980586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:19.994307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1328/2670] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:25:19.998664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c503cbf-da38-43d8-a368-976451593d55 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.008326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c503cbf-da38-43d8-a368-976451593d55 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.013154 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.014072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.014363 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.014642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.015338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.018224 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.047878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1343/2671] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:19 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:20.052586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.063023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb9860-79a3-4007-9802-843ef1220c18 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.095983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:20.125152 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.125945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.126247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.126502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.127243 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.129213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.150204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1329/2672] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:20 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:20.154327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.165088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4951a-d198-4ad1-8fdd-5cb28c4abbd4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.169674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.170376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.170666 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.170949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.171681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.173820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.184170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1344/2673] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:20.188546 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.198334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8becf598-910e-4a1c-bd57-93554f291ff5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.202876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08349b0c-0ac0-4e5d-91a6-6f1dd07e4336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.203526 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08349b0c-0ac0-4e5d-91a6-6f1dd07e4336 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.203812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08349b0c-0ac0-4e5d-91a6-6f1dd07e4336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.204091 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08349b0c-0ac0-4e5d-91a6-6f1dd07e4336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.204811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08349b0c-0ac0-4e5d-91a6-6f1dd07e4336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.206523 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08349b0c-0ac0-4e5d-91a6-6f1dd07e4336 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.216786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1330/2674] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:20.220660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.230965 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13099110-94e7-4679-bfda-98f91581dee9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.236078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.236941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.237219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.237489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] PATH_INFO: `/v3/projects/a4902d01c5044cad97843dd5f4742de9/users/db1d1606f20a4807b52756f2169b5fe4/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.252340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4902d01c5044cad97843dd5f4742de9, user_id=db1d1606f20a4807b52756f2169b5fe4, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.255032 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.267538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1345/2675] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:20 2022] PUT /identity/v3/projects/a4902d01c5044cad97843dd5f4742de9/users/db1d1606f20a4807b52756f2169b5fe4/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:20.272479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e17b3-17c5-4795-9da0-f5aa6376a525 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.272775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e17b3-17c5-4795-9da0-f5aa6376a525 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.273056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e17b3-17c5-4795-9da0-f5aa6376a525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.290074 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a43e17b3-17c5-4795-9da0-f5aa6376a525 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:20.291258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a43e17b3-17c5-4795-9da0-f5aa6376a525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:20.350364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1331/2676] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:20.360395 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:20.362168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:20.400113 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:20.440715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a412-94c6-401a-89c4-2e7225c57ab1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:20.444409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:20.445054 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:20.445337 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:20.445613 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:20.455034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:20.456477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:20.466233 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1346/2677] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:21.064590 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:21.065102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:21.065841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:21.096521 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] MFA Rules not processed for user `db1d1606f20a4807b52756f2169b5fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:21.098108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:21.178180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1332/2678] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:21.190652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:21.195922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:21.267799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5764c-d369-439d-9863-be39aedcdc7f service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:21.272329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad3387-942e-4bd9-8024-4ec0437c1d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:21.273463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad3387-942e-4bd9-8024-4ec0437c1d73 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:21.273914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad3387-942e-4bd9-8024-4ec0437c1d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:21.274367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad3387-942e-4bd9-8024-4ec0437c1d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:21.283793 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad3387-942e-4bd9-8024-4ec0437c1d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:21.285228 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad3387-942e-4bd9-8024-4ec0437c1d73 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:21.294078 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1347/2679] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:22.277084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:22.278508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.292181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fc787b-0f1e-4ebe-a2b2-6dc5bbf7fbab None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:22.296708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:22.298180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.298468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.298748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] PATH_INFO: `/v3/users/382086525a1d4e39bbf01e69524cb504` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.299931 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382086525a1d4e39bbf01e69524cb504)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:22.301867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:22.338619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:22.348563 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382086525a1d4e39bbf01e69524cb504'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.349541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382086525a1d4e39bbf01e69524cb504'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.356793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382086525a1d4e39bbf01e69524cb504'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.357433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382086525a1d4e39bbf01e69524cb504'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.365163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382086525a1d4e39bbf01e69524cb504'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:22.382607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1333/2680] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:22 2022] DELETE /identity/v3/users/382086525a1d4e39bbf01e69524cb504 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:22.387178 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cca0f-857f-48b2-af41-e1f58d39ec93 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.387661 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cca0f-857f-48b2-af41-e1f58d39ec93 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.387781 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cca0f-857f-48b2-af41-e1f58d39ec93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.405037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-366cca0f-857f-48b2-af41-e1f58d39ec93 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:22.406244 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-366cca0f-857f-48b2-af41-e1f58d39ec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.466591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1348/2681] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:22.477245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:22.478833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.535707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ea533-2514-4d3c-a487-845e84932c29 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:22.539421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:22.540066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.540354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.540635 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.550785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:22.552912 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:22.562793 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1334/2682] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:22.652277 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.652575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.652862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.671253 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] MFA Rules not processed for user `db1d1606f20a4807b52756f2169b5fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:22.672893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.730915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1349/2683] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:22.975210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:22.976728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:22.987984 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c7190-3c2c-4be7-bea5-3d0d233bce09 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:22.991675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d4204-7094-49b9-aad4-f49a35c6854f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:22.992309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d4204-7094-49b9-aad4-f49a35c6854f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:22.992592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d4204-7094-49b9-aad4-f49a35c6854f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:22.992871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d4204-7094-49b9-aad4-f49a35c6854f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:22.994619 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8c6d4204-7094-49b9-aad4-f49a35c6854f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.004583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d4204-7094-49b9-aad4-f49a35c6854f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.006039 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d4204-7094-49b9-aad4-f49a35c6854f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.014289 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1335/2684] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:23.063421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.065159 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.081535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d26e5-f6e3-4ad1-afba-9cd052b421f3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.086334 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.087210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.087502 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.087914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] PATH_INFO: `/v3/projects/6293b2fd55344b2e9fe316677d5beff4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.089292 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6293b2fd55344b2e9fe316677d5beff4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.091736 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.094727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:23.150107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6293b2fd55344b2e9fe316677d5beff4', 'request_id': 'req-17d7caee-f8d2-44ad-a39f-63c96f929a4a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:23.159196 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6293b2fd55344b2e9fe316677d5beff4', 'request_id': 'req-17d7caee-f8d2-44ad-a39f-63c96f929a4a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:23.165955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6293b2fd55344b2e9fe316677d5beff4', 'request_id': 'req-17d7caee-f8d2-44ad-a39f-63c96f929a4a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:23.176354 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] The token cache is being invalidate because project 6293b2fd55344b2e9fe316677d5beff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:25:23.176595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:23.177222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17d7caee-f8d2-44ad-a39f-63c96f929a4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:25:23.178805 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1350/2685] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:25:23 2022] DELETE /identity/v3/projects/6293b2fd55344b2e9fe316677d5beff4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:23.183771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1a873-fbd2-494a-82f1-7d166fff3858 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.184064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1a873-fbd2-494a-82f1-7d166fff3858 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.184344 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1a873-fbd2-494a-82f1-7d166fff3858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.202122 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c1b1a873-fbd2-494a-82f1-7d166fff3858 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:23.203308 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c1b1a873-fbd2-494a-82f1-7d166fff3858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.265823 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1336/2686] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:23.275089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e5a9-3c40-4a48-ac5c-51431dc1d9c0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.275385 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e5a9-3c40-4a48-ac5c-51431dc1d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.275670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808e5a9-3c40-4a48-ac5c-51431dc1d9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.293872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a808e5a9-3c40-4a48-ac5c-51431dc1d9c0 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:23.295074 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a808e5a9-3c40-4a48-ac5c-51431dc1d9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.304093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1351/2687] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:23.308914 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1a873-fbd2-494a-82f1-7d166fff3858 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.319426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1a873-fbd2-494a-82f1-7d166fff3858 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.324545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.325219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.325503 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.325817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.326543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:25:23.326849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.342743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.357756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1337/2688] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:25:23.362137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e5a9-3c40-4a48-ac5c-51431dc1d9c0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.372559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808e5a9-3c40-4a48-ac5c-51431dc1d9c0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.379563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.380236 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.380669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.380949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.381656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.383587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.413123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1352/2689] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:23.417547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.431132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2adb2-a8c0-406b-98ec-b795e9365cf3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.461236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:23.493903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.494778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.495069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.495522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.496395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.498472 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.522508 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1338/2690] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:25:23.524360 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.535784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0590b77-9df6-41ce-b7ba-255f6254b087 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.541104 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.542000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.542705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.543121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.544028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.545633 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.560992 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1353/2691] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:23.562318 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.573554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5209ed-8eeb-47ba-808e-5f68bf94b349 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.579082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc5fab-8dc4-4686-8771-acdd1bc3351f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.579748 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc5fab-8dc4-4686-8771-acdd1bc3351f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.580035 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc5fab-8dc4-4686-8771-acdd1bc3351f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.580315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc5fab-8dc4-4686-8771-acdd1bc3351f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.581105 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc5fab-8dc4-4686-8771-acdd1bc3351f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.582695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc5fab-8dc4-4686-8771-acdd1bc3351f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.592935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1339/2692] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:25:23.597925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.608798 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e5d40-d229-482e-a166-66e1a9a66533 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.614622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.615607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.615947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.616219 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] PATH_INFO: `/v3/projects/1c57c581f6084942b922c85ecd5bfd89/users/2e7a1318e2ef460591e7b922bb848257/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.632950 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c57c581f6084942b922c85ecd5bfd89, user_id=2e7a1318e2ef460591e7b922bb848257, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.635804 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.650357 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1354/2693] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:25:23 2022] PUT /identity/v3/projects/1c57c581f6084942b922c85ecd5bfd89/users/2e7a1318e2ef460591e7b922bb848257/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:25:23.660839 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.661134 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.661431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.689845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] MFA Rules not processed for user `2e7a1318e2ef460591e7b922bb848257`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:23.691082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.758504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1340/2694] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:23.770001 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:23.771702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:23.831951 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572722c-1bff-4dfc-b914-16f7d81b931c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:23.835962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7a80a-564e-42b1-b17f-4360a615a012 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:23.836620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7a80a-564e-42b1-b17f-4360a615a012 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:23.836911 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7a80a-564e-42b1-b17f-4360a615a012 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:23.837223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7a80a-564e-42b1-b17f-4360a615a012 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:23.846507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7a80a-564e-42b1-b17f-4360a615a012 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:23.848202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7a80a-564e-42b1-b17f-4360a615a012 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:23.857212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1355/2695] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:32.818408 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:32.819974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:32.931561 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e55988-8e90-46f7-b6b5-6c0cc3dff7d7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:32.938250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:32.938882 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:32.939164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:32.939436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:32.941205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:32.955929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:32.957566 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:32.962595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:32.970225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1341/2696] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:33.853057 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:39.907323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda41b-0cb9-4ccf-85f7-c82e15b50fe6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:39.907898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda41b-0cb9-4ccf-85f7-c82e15b50fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:39.907898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda41b-0cb9-4ccf-85f7-c82e15b50fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:39.925004 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-69bda41b-0cb9-4ccf-85f7-c82e15b50fe6 None None] MFA Rules not processed for user `db1d1606f20a4807b52756f2169b5fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:25:39.926243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-69bda41b-0cb9-4ccf-85f7-c82e15b50fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:39.979977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1356/2697] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:25:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:25:40.209974 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:25:40.211387 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:40.267695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc59078-216b-4300-a2a1-24560dbe7dc2 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:25:40.271313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3872dbe-e580-46d0-ae75-660230ac1788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:25:40.271947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3872dbe-e580-46d0-ae75-660230ac1788 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:25:40.272232 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3872dbe-e580-46d0-ae75-660230ac1788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:25:40.272509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3872dbe-e580-46d0-ae75-660230ac1788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:25:40.274273 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f3872dbe-e580-46d0-ae75-660230ac1788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:25:40.284111 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3872dbe-e580-46d0-ae75-660230ac1788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:25:40.285527 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3872dbe-e580-46d0-ae75-660230ac1788 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:25:40.293756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1342/2698] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:25:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:25:49.973516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=25 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:50.289989 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=27 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:25:59.973886 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda41b-0cb9-4ccf-85f7-c82e15b50fe6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:00.037586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda41b-0cb9-4ccf-85f7-c82e15b50fe6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:00.042225 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:00.043201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.043521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.043883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] PATH_INFO: `/v3/users/2e7a1318e2ef460591e7b922bb848257` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.045088 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7a1318e2ef460591e7b922bb848257)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:00.047552 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:00.095681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7a1318e2ef460591e7b922bb848257'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.096261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7a1318e2ef460591e7b922bb848257'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.097124 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7a1318e2ef460591e7b922bb848257'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.106165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7a1318e2ef460591e7b922bb848257'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.114189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7a1318e2ef460591e7b922bb848257'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.132662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1357/2699] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:25:59 2022] DELETE /identity/v3/users/2e7a1318e2ef460591e7b922bb848257 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:00.138022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.138022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.138022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.156432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:00.157889 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:00.224659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1343/2700] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:00.234171 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:00.235601 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:00.310913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ed3e-2235-468c-aa5a-9fc1890c13fc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:00.314820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b58a1-64ed-4c13-b087-40106d4633c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:00.315551 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b58a1-64ed-4c13-b087-40106d4633c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.315917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b58a1-64ed-4c13-b087-40106d4633c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.316263 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b58a1-64ed-4c13-b087-40106d4633c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.326672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b58a1-64ed-4c13-b087-40106d4633c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:00.328663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b58a1-64ed-4c13-b087-40106d4633c9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:00.338695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1358/2701] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:00.609587 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:00.611431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:00.632129 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea49a9-66df-4f0c-9ba1-4edc49426f35 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:00.636957 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:00.638409 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.638698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.639006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] PATH_INFO: `/v3/projects/1c57c581f6084942b922c85ecd5bfd89` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.640849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c57c581f6084942b922c85ecd5bfd89)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:00.642751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:00.692606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c57c581f6084942b922c85ecd5bfd89', 'request_id': 'req-a65f28af-402c-44f9-a1cc-c7a5b4456d94'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.700280 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c57c581f6084942b922c85ecd5bfd89', 'request_id': 'req-a65f28af-402c-44f9-a1cc-c7a5b4456d94'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.709081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c57c581f6084942b922c85ecd5bfd89', 'request_id': 'req-a65f28af-402c-44f9-a1cc-c7a5b4456d94'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.720951 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] The token cache is being invalidate because project 1c57c581f6084942b922c85ecd5bfd89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:26:00.721236 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.722200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a65f28af-402c-44f9-a1cc-c7a5b4456d94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:00.723724 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1344/2702] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:26:00 2022] DELETE /identity/v3/projects/1c57c581f6084942b922c85ecd5bfd89 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:00.728785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5133dc-7fcc-42f8-9573-35602f46a6cb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.729084 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5133dc-7fcc-42f8-9573-35602f46a6cb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.729373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5133dc-7fcc-42f8-9573-35602f46a6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.747169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4d5133dc-7fcc-42f8-9573-35602f46a6cb None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:00.748383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4d5133dc-7fcc-42f8-9573-35602f46a6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:00.768433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:00.814218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1359/2703] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:00.822022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135718ca-8987-4024-8fb1-1bb58900cd61 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.822315 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135718ca-8987-4024-8fb1-1bb58900cd61 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.822597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135718ca-8987-4024-8fb1-1bb58900cd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.842217 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-135718ca-8987-4024-8fb1-1bb58900cd61 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:00.843592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-135718ca-8987-4024-8fb1-1bb58900cd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:00.852844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1345/2704] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:00.857308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5133dc-7fcc-42f8-9573-35602f46a6cb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:00.869248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5133dc-7fcc-42f8-9573-35602f46a6cb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:00.874585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:00.875262 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.875556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.875839 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.876573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:00.876891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:00.878537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:00.897750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1360/2705] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:00.898856 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135718ca-8987-4024-8fb1-1bb58900cd61 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:00.908773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135718ca-8987-4024-8fb1-1bb58900cd61 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:00.913578 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:00.914376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:00.914657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:00.914928 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:00.915609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:00.917491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:00.951768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1346/2706] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:26:00 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:26:00.955940 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:00.966069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae8dd0-24ea-4ea2-94b3-9ff44015c5af admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:01.021970 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:01.022623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:01.022913 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:01.023193 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:01.023900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:01.025842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:01.046355 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1361/2707] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:26:00 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:01.050593 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:01.060920 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a82ad9-409c-40de-8728-743a5ad64b74 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:01.066079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:01.066733 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:01.067018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:01.067296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:01.068011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:01.069591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:01.080887 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1347/2708] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:26:01.084974 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:01.094953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ab874-7098-4134-b2e5-45ea70917069 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:01.099876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e236a98-b12c-4261-bc60-cbf1ffb20cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:01.100532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e236a98-b12c-4261-bc60-cbf1ffb20cbd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:01.100820 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e236a98-b12c-4261-bc60-cbf1ffb20cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:01.101100 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e236a98-b12c-4261-bc60-cbf1ffb20cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:01.101865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e236a98-b12c-4261-bc60-cbf1ffb20cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:01.103458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e236a98-b12c-4261-bc60-cbf1ffb20cbd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:01.114042 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1362/2709] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:26:01.118063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:01.128515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049fbed-6a60-4dc6-b323-420e402e667d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:01.133428 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:01.134351 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:01.134767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:01.135073 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] PATH_INFO: `/v3/projects/995d839e52d24415b625f6f59f3af4fc/users/b1178e82a3e6424099bc5d11e6cf6227/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:01.152241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=995d839e52d24415b625f6f59f3af4fc, user_id=b1178e82a3e6424099bc5d11e6cf6227, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:01.155077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:01.168055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1348/2710] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:26:01 2022] PUT /identity/v3/projects/995d839e52d24415b625f6f59f3af4fc/users/b1178e82a3e6424099bc5d11e6cf6227/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:01.232134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a5ae-15b3-41f2-a6bb-3a54ad4e0600 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:01.232453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a5ae-15b3-41f2-a6bb-3a54ad4e0600 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:01.232754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2a5ae-15b3-41f2-a6bb-3a54ad4e0600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:01.260941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-91e2a5ae-15b3-41f2-a6bb-3a54ad4e0600 None None] MFA Rules not processed for user `b1178e82a3e6424099bc5d11e6cf6227`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:01.262371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-91e2a5ae-15b3-41f2-a6bb-3a54ad4e0600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:01.329651 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1363/2711] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:01.340826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:01.342547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:01.366016 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:01.424771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156a890-ee2c-413c-a83d-f0df2929c266 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:01.428416 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:01.429131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:01.429417 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:01.429692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:01.438780 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:01.440200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:01.449436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1349/2712] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:08.335373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367200d0-d39e-423e-8925-b1ca990b1256 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:08.335871 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367200d0-d39e-423e-8925-b1ca990b1256 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:08.335976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367200d0-d39e-423e-8925-b1ca990b1256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:08.361402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:08.395578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-367200d0-d39e-423e-8925-b1ca990b1256 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:08.396723 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-367200d0-d39e-423e-8925-b1ca990b1256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:08.462006 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1364/2713] 10.222.0.88 () {58 vars in 1009 bytes} [Thu Jun 16 16:26:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:08.511296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:08.512695 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:08.611300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89775b63-baf2-41e2-980c-899c7cee9a64 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:08.616755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:08.617411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:08.617698 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:08.618008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:08.628930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:08.630688 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:08.639558 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1350/2714] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:26:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:26:18.458507 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=36 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:18.636753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=67 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:25.783123 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.783639 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.783756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.802453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] MFA Rules not processed for user `bf00375b0f334f7bada8f21ea3766ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:25.803638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:25.811815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1365/2715] 10.222.0.88 () {58 vars in 967 bytes} [Thu Jun 16 16:26:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:25.828738 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:25.839440 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c7a6b-f531-47fb-be97-54c94e411469 service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:25.844461 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b1f0775ba4875a550ea986ed483e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:25.845100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:25.845384 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:25.845692 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:25.854076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:25.855647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:25.864430 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1351/2716] 10.222.0.88 () {60 vars in 1384 bytes} [Thu Jun 16 16:26:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 16:26:27.715526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:27.717015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:27.774059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:27.775900 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:27.819140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392b9d7-b4fd-4669-ade7-65fec413c9f1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:27.825787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:27.826585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:27.826967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:27.827441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:27.829514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:27.842400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:27.844132 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:27.849257 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6554c13-e870-411a-a7d2-5f3c22bb3171 service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:27.853854 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:27.854741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:27.855005 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1366/2717] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:27.855124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:27.855374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] PATH_INFO: `/v3/users/db1d1606f20a4807b52756f2169b5fe4` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:27.856642 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1d1606f20a4807b52756f2169b5fe4)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:27.859132 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:27.909443 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1d1606f20a4807b52756f2169b5fe4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:27.910099 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1d1606f20a4807b52756f2169b5fe4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:27.917011 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1d1606f20a4807b52756f2169b5fe4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:27.917623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1d1606f20a4807b52756f2169b5fe4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:27.925876 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dbbaeb05-1a75-416d-b5ac-b6fa871ecaa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1d1606f20a4807b52756f2169b5fe4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:27.944075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1352/2718] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:26:27 2022] DELETE /identity/v3/users/db1d1606f20a4807b52756f2169b5fe4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:26:28.118765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.136669 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c6778-21cd-4f99-8dd7-aaa30acc288e service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.195579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.196453 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.196735 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.197034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] PATH_INFO: `/v3/projects/a4902d01c5044cad97843dd5f4742de9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.198227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4902d01c5044cad97843dd5f4742de9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.200177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.249470 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4902d01c5044cad97843dd5f4742de9', 'request_id': 'req-bddc9b61-f8a5-411d-bca9-a245f497ad94'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:28.257514 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4902d01c5044cad97843dd5f4742de9', 'request_id': 'req-bddc9b61-f8a5-411d-bca9-a245f497ad94'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:28.263790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4902d01c5044cad97843dd5f4742de9', 'request_id': 'req-bddc9b61-f8a5-411d-bca9-a245f497ad94'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:28.267622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:28.276417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] The token cache is being invalidate because project a4902d01c5044cad97843dd5f4742de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:26:28.276976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:28.277960 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bddc9b61-f8a5-411d-bca9-a245f497ad94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:28.280433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1367/2719] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:26:28 2022] DELETE /identity/v3/projects/a4902d01c5044cad97843dd5f4742de9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:28.285376 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b623e-add2-423a-9fa0-8e230685ca81 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.285668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b623e-add2-423a-9fa0-8e230685ca81 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.285991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b623e-add2-423a-9fa0-8e230685ca81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.304227 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a33b623e-add2-423a-9fa0-8e230685ca81 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:28.305354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a33b623e-add2-423a-9fa0-8e230685ca81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:28.366874 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1353/2720] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:28.374817 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32459-30ee-4b62-9d77-3169239c6eea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.375108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32459-30ee-4b62-9d77-3169239c6eea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.375389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a32459-30ee-4b62-9d77-3169239c6eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.394274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-09a32459-30ee-4b62-9d77-3169239c6eea None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:28.395477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-09a32459-30ee-4b62-9d77-3169239c6eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:28.403731 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1368/2721] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:28.407504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b623e-add2-423a-9fa0-8e230685ca81 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.417660 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b623e-add2-423a-9fa0-8e230685ca81 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.422314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.422977 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.423263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.423539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.424296 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:28.424604 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.426248 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.427904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:28.440781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1354/2722] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:28.445313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a32459-30ee-4b62-9d77-3169239c6eea None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.455210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a32459-30ee-4b62-9d77-3169239c6eea None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.459784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.460432 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.460715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.460990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.461679 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.463612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.490863 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1369/2723] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:26:28 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:28.495531 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.507795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355e6e5-c91d-4e5a-82ef-aad5989363c4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.567656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.568400 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.568723 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.569191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.570148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.572399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.594808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1355/2724] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:26:28 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:28.601049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.610657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c482a6-37f5-4593-9d21-97b3e357d0fb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.615725 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.616480 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.616848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.617200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.618047 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.619652 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.631303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1370/2725] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:26:28.635264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.645929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b292db-7dd4-4b1f-8998-1b9865026932 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.651365 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b8b81-acfe-45c6-be5c-fd7fd30c9ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.652291 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b8b81-acfe-45c6-be5c-fd7fd30c9ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.652722 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b8b81-acfe-45c6-be5c-fd7fd30c9ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.653061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b8b81-acfe-45c6-be5c-fd7fd30c9ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.654404 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b8b81-acfe-45c6-be5c-fd7fd30c9ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.656138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b8b81-acfe-45c6-be5c-fd7fd30c9ec0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.666650 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1356/2726] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:26:28.670782 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.681107 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee409af-5cc7-41e4-a9e1-d3ba88e6fc6f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.686221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.687134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.687413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.687683 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] PATH_INFO: `/v3/projects/e0ecc548719b49d4b75b13b887ffcbf7/users/84e9ddbee47949c9a9cd195ce4749041/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.703508 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ecc548719b49d4b75b13b887ffcbf7, user_id=84e9ddbee47949c9a9cd195ce4749041, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.706411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.721010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1371/2727] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:26:28 2022] PUT /identity/v3/projects/e0ecc548719b49d4b75b13b887ffcbf7/users/84e9ddbee47949c9a9cd195ce4749041/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:28.731649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.732148 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.732484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.761484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] MFA Rules not processed for user `84e9ddbee47949c9a9cd195ce4749041`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:28.762820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:28.830184 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1357/2728] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:28.841602 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:28.843737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:28.923254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d677b0-f784-4dae-9d20-47152cadeb00 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:28.927150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:28.927809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:28.928101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:28.928389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:28.937529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:28.939106 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:28.947862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1372/2729] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:38.827138 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:38.943899 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=1,keystone:SELECT=64,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:40.462594 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:40.464002 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:40.538850 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da790f67-988e-43c3-b1db-7edd1730e021 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:40.543943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2c8f3-a8ce-40f7-b765-f2293e9acd68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:40.545124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2c8f3-a8ce-40f7-b765-f2293e9acd68 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:40.545446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2c8f3-a8ce-40f7-b765-f2293e9acd68 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:40.545746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2c8f3-a8ce-40f7-b765-f2293e9acd68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:40.547539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-72b2c8f3-a8ce-40f7-b765-f2293e9acd68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:40.559519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2c8f3-a8ce-40f7-b765-f2293e9acd68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:40.561119 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2c8f3-a8ce-40f7-b765-f2293e9acd68 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:40.571019 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1358/2730] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:50.566734 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=32 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:55.599413 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.600865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:55.666586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e03ec-d047-4795-9032-dc61dfac0945 service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.671582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.672444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.672730 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.673007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] PATH_INFO: `/v3/users/b1178e82a3e6424099bc5d11e6cf6227` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.674207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1178e82a3e6424099bc5d11e6cf6227)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.676126 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.720907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1178e82a3e6424099bc5d11e6cf6227'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:55.721843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1178e82a3e6424099bc5d11e6cf6227'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:55.722458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1178e82a3e6424099bc5d11e6cf6227'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:55.729800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1178e82a3e6424099bc5d11e6cf6227'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:55.737202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1178e82a3e6424099bc5d11e6cf6227'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:55.756349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1373/2731] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:26:55 2022] DELETE /identity/v3/users/b1178e82a3e6424099bc5d11e6cf6227 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:55.760811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.761267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.761379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.778213 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:55.779827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:55.838774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1359/2732] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:55.849343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:55.850785 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:55.921238 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd274a-1905-49ca-b84b-2df38f7714d2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:55.924826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafdd1b-7748-45cf-96b7-1270035361d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:55.925468 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafdd1b-7748-45cf-96b7-1270035361d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:55.925778 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafdd1b-7748-45cf-96b7-1270035361d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:55.926059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafdd1b-7748-45cf-96b7-1270035361d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:55.935081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafdd1b-7748-45cf-96b7-1270035361d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:55.936522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafdd1b-7748-45cf-96b7-1270035361d1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:55.946534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1374/2733] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:26:56.224321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.226207 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:56.238905 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bb5f8-d0b2-4651-85f7-d5850a1982b9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.244054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.245298 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.245584 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.246965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] PATH_INFO: `/v3/projects/995d839e52d24415b625f6f59f3af4fc` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.247656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=995d839e52d24415b625f6f59f3af4fc)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.249834 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.298638 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '995d839e52d24415b625f6f59f3af4fc', 'request_id': 'req-0c9fc15a-a19c-4d04-9fd4-8228eee80248'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:56.305668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '995d839e52d24415b625f6f59f3af4fc', 'request_id': 'req-0c9fc15a-a19c-4d04-9fd4-8228eee80248'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:56.312477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '995d839e52d24415b625f6f59f3af4fc', 'request_id': 'req-0c9fc15a-a19c-4d04-9fd4-8228eee80248'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:56.323568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] The token cache is being invalidate because project 995d839e52d24415b625f6f59f3af4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:26:56.323857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:56.324513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0c9fc15a-a19c-4d04-9fd4-8228eee80248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:26:56.326669 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1360/2734] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:26:56 2022] DELETE /identity/v3/projects/995d839e52d24415b625f6f59f3af4fc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:26:56.331754 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d174d1-e043-4087-a7c9-82736e41f11d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.332049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d174d1-e043-4087-a7c9-82736e41f11d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.332333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d174d1-e043-4087-a7c9-82736e41f11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.349895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-92d174d1-e043-4087-a7c9-82736e41f11d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:56.351019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92d174d1-e043-4087-a7c9-82736e41f11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:56.368085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:56.411268 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1375/2735] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:56.419603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcc49-308e-420a-a1d8-c006f955341e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.419927 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcc49-308e-420a-a1d8-c006f955341e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.420215 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcc49-308e-420a-a1d8-c006f955341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.438908 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3b5dcc49-308e-420a-a1d8-c006f955341e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:56.440103 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3b5dcc49-308e-420a-a1d8-c006f955341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:56.447945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1361/2736] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:56.452138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d174d1-e043-4087-a7c9-82736e41f11d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.461815 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d174d1-e043-4087-a7c9-82736e41f11d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.466276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.466925 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.467213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.467496 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.468221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:26:56.468535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.470169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.482279 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1376/2737] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:26:56.486368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcc49-308e-420a-a1d8-c006f955341e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.496810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcc49-308e-420a-a1d8-c006f955341e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.501494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.502179 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.502468 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.502751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.503434 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.505335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.531776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1362/2738] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:26:56 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:56.536058 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.546131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2305fec9-21c7-4171-937c-3609608eb07a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.602203 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.602843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.603076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.603361 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.604061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.606024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.626571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1377/2739] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:26:56 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:26:56.630399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.641313 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc01e0-df01-4013-a93c-c6e0255c1633 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.646372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.647062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.647349 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.647903 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.648366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.650005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.661429 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1363/2740] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:26:56.665433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.675870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328db86-91c7-4a7e-901d-9c2807a1cdf5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.680475 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc99740-76ef-4ac5-809d-ceaffb648187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.681131 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc99740-76ef-4ac5-809d-ceaffb648187 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.681417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc99740-76ef-4ac5-809d-ceaffb648187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.681698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc99740-76ef-4ac5-809d-ceaffb648187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.682458 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc99740-76ef-4ac5-809d-ceaffb648187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.684081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc99740-76ef-4ac5-809d-ceaffb648187 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.693924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1378/2741] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:26:56.697960 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.708283 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c31342-e6ab-4cd0-8e6c-dc79d664ebe6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.712843 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.713768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.714058 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.714337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] PATH_INFO: `/v3/projects/7c94dc0baa2d4a808cfbcf090014e8fe/users/37eb761516834a2aa633fb8a4c19dee3/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.730929 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c94dc0baa2d4a808cfbcf090014e8fe, user_id=37eb761516834a2aa633fb8a4c19dee3, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.733616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.746952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1364/2742] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:26:56 2022] PUT /identity/v3/projects/7c94dc0baa2d4a808cfbcf090014e8fe/users/37eb761516834a2aa633fb8a4c19dee3/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:26:56.757989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac049b9-9063-4e23-91bc-092c6d904b62 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.758281 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac049b9-9063-4e23-91bc-092c6d904b62 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.758559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac049b9-9063-4e23-91bc-092c6d904b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.784727 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4ac049b9-9063-4e23-91bc-092c6d904b62 None None] MFA Rules not processed for user `37eb761516834a2aa633fb8a4c19dee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:26:56.785955 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ac049b9-9063-4e23-91bc-092c6d904b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:56.850192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1379/2743] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:26:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:26:56.860945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:26:56.862800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:26:56.885744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:26:56.922934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0660c-0d67-4860-8ac8-60e158ec8450 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:26:56.926639 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:26:56.927284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:26:56.927570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:26:56.927851 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:26:56.936509 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:26:56.938070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:26:56.947197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1365/2744] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:26:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:27:06.847218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:06.943067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=17 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:17.372201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac049b9-9063-4e23-91bc-092c6d904b62 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:17.381932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac049b9-9063-4e23-91bc-092c6d904b62 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:17.385420 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea28b0-b861-4141-9eda-fd2dd1da89b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:17.386090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea28b0-b861-4141-9eda-fd2dd1da89b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:17.386438 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea28b0-b861-4141-9eda-fd2dd1da89b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:17.387310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea28b0-b861-4141-9eda-fd2dd1da89b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:17.389071 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c7ea28b0-b861-4141-9eda-fd2dd1da89b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:17.445524 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea28b0-b861-4141-9eda-fd2dd1da89b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:17.446995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea28b0-b861-4141-9eda-fd2dd1da89b1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:17.456980 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1380/2745] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:27:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:27:23.639934 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:23.641451 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:23.708031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee72051-95be-4402-b385-35b1ee2f548f service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:23.712355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:23.713237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:23.713596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:23.714211 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:23.716484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:23.829774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:23.831362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:23.844709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1366/2746] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:27:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:27:27.452664 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:29.273332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a67f43-61f3-4d15-99be-af594374ab10 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.273922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a67f43-61f3-4d15-99be-af594374ab10 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.273922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a67f43-61f3-4d15-99be-af594374ab10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.291140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-19a67f43-61f3-4d15-99be-af594374ab10 None None] MFA Rules not processed for user `c6c53bf1554a4825afd8770ea9347ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:29.292842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-19a67f43-61f3-4d15-99be-af594374ab10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:29.299587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1381/2747] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:27:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:27:29.335272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:29.336676 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:29.392364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf88909-e6f9-4d1b-9138-59b8c27ecc23 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:29.395909 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d43d4-2128-4b84-b545-b3a8155e6950 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982fcdf935f64b98aae10f206a21da2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:29.396552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d43d4-2128-4b84-b545-b3a8155e6950 service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:29.396841 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d43d4-2128-4b84-b545-b3a8155e6950 service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:29.397171 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d43d4-2128-4b84-b545-b3a8155e6950 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:29.399064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-093d43d4-2128-4b84-b545-b3a8155e6950 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:29.412794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d43d4-2128-4b84-b545-b3a8155e6950 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:29.414253 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d43d4-2128-4b84-b545-b3a8155e6950 service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:29.425678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1367/2748] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:27:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:27:33.776794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a67f43-61f3-4d15-99be-af594374ab10 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:33.840393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a67f43-61f3-4d15-99be-af594374ab10 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:33.845059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:33.845963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.846510 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.846794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] PATH_INFO: `/v3/users/37eb761516834a2aa633fb8a4c19dee3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.847986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37eb761516834a2aa633fb8a4c19dee3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:33.850021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:33.896819 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37eb761516834a2aa633fb8a4c19dee3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:33.897463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37eb761516834a2aa633fb8a4c19dee3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:33.898081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37eb761516834a2aa633fb8a4c19dee3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:33.905671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37eb761516834a2aa633fb8a4c19dee3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:33.912505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37eb761516834a2aa633fb8a4c19dee3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:33.929997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1382/2749] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:27:33 2022] DELETE /identity/v3/users/37eb761516834a2aa633fb8a4c19dee3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:27:33.934320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69286084-b41e-4935-a087-3d476d97f20f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:33.934826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69286084-b41e-4935-a087-3d476d97f20f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:33.934954 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69286084-b41e-4935-a087-3d476d97f20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:33.951771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-69286084-b41e-4935-a087-3d476d97f20f None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:33.952973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69286084-b41e-4935-a087-3d476d97f20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:33.995592 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:34.014452 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1368/2750] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:27:34.024172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.025576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:34.085120 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dea80-5b82-4ef5-ad1e-cbb82f6895f4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.088751 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.089944 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.090230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.090526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.099657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.101092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.110864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1383/2751] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:27:34.383845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69286084-b41e-4935-a087-3d476d97f20f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:34.385329 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69286084-b41e-4935-a087-3d476d97f20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:34.403852 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69286084-b41e-4935-a087-3d476d97f20f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:34.409052 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:34.409962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:34.410277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:34.410554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] PATH_INFO: `/v3/projects/7c94dc0baa2d4a808cfbcf090014e8fe` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:34.411759 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c94dc0baa2d4a808cfbcf090014e8fe)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:34.413795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:34.469812 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c94dc0baa2d4a808cfbcf090014e8fe', 'request_id': 'req-f0426579-10c9-4013-8488-9a78ef3a0d38'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:34.477366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c94dc0baa2d4a808cfbcf090014e8fe', 'request_id': 'req-f0426579-10c9-4013-8488-9a78ef3a0d38'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:34.484223 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c94dc0baa2d4a808cfbcf090014e8fe', 'request_id': 'req-f0426579-10c9-4013-8488-9a78ef3a0d38'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:34.494663 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] The token cache is being invalidate because project 7c94dc0baa2d4a808cfbcf090014e8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:27:34.495519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:34.496170 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f0426579-10c9-4013-8488-9a78ef3a0d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:34.497712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1369/2752] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:27:34 2022] DELETE /identity/v3/projects/7c94dc0baa2d4a808cfbcf090014e8fe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:36.917546 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:36.919134 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:36.931587 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf910-c79c-4a02-b196-e27a5c79e4d4 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:36.944102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:36.992645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:36.993640 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:36.993962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:36.994247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] PATH_INFO: `/v3/users/84e9ddbee47949c9a9cd195ce4749041` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:36.995935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e9ddbee47949c9a9cd195ce4749041)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:36.997900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.048265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e9ddbee47949c9a9cd195ce4749041'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.048922 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e9ddbee47949c9a9cd195ce4749041'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.049532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e9ddbee47949c9a9cd195ce4749041'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.056880 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e9ddbee47949c9a9cd195ce4749041'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.064309 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e9ddbee47949c9a9cd195ce4749041'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.082294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1384/2753] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:27:36 2022] DELETE /identity/v3/users/84e9ddbee47949c9a9cd195ce4749041 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:37.087836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.088385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.088871 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.108229 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:27:37.109575 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:37.178272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1370/2754] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:27:37.188366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.192878 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:37.246576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a291e9a-cafa-4e55-8854-094bb655acbe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.250188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.250864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.251149 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.251427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.260844 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.262316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.271582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1385/2755] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:27:37.537018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:27:37.538603 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:27:37.553501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41f01e-bdc6-435b-a2bd-3728333657d7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:27:37.558241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:27:37.559239 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:27:37.559536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:27:37.559808 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] PATH_INFO: `/v3/projects/e0ecc548719b49d4b75b13b887ffcbf7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:27:37.561043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ecc548719b49d4b75b13b887ffcbf7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:27:37.563029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:27:37.614990 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ecc548719b49d4b75b13b887ffcbf7', 'request_id': 'req-9b1fadd1-af87-437d-bd8c-a39de6303828'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.622701 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ecc548719b49d4b75b13b887ffcbf7', 'request_id': 'req-9b1fadd1-af87-437d-bd8c-a39de6303828'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.628983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ecc548719b49d4b75b13b887ffcbf7', 'request_id': 'req-9b1fadd1-af87-437d-bd8c-a39de6303828'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.631027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:37.642725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] The token cache is being invalidate because project e0ecc548719b49d4b75b13b887ffcbf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:27:37.643013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.643730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:27:37.645260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1371/2756] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:27:37 2022] DELETE /identity/v3/projects/e0ecc548719b49d4b75b13b887ffcbf7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:27:47.267443 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:27:47.640965 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:28:41.130445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:28:41.141255 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db260c78-9520-4617-acaa-d433105e5e32 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:28:41.188682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:28:41.189339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:28:41.189625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:28:41.189933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:28:41.191753 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:28:41.201741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:28:41.203214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:28:41.211251 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1386/2757] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:28:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:28:51.207518 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=26 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:35.176460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:35.178226 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:35.294556 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fadd1-af87-437d-bd8c-a39de6303828 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:35.301978 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:35.303648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:35.304659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:35.305124 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:35.308356 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:35.327785 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:35.330247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:35.359028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1372/2758] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:40.988997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:40.990658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:41.003841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3700e-ada2-4fc0-8ca0-9f64a0cbd942 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:41.008643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:41.009417 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:41.009784 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:41.010143 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:41.012265 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:41.088208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:41.090079 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:41.104381 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1387/2759] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:44.406398 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:44.407897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:44.471998 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:44.474220 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:44.487750 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610869a4-2687-4155-b417-0b0c8cbbcda8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:44.492662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:44.493564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:44.493873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:44.494466 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] PATH_INFO: `/v3/users/c6c53bf1554a4825afd8770ea9347ad1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:44.495331 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da8f7d-25ee-4914-b641-4b206390ad41 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:44.495662 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c53bf1554a4825afd8770ea9347ad1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:44.497591 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:44.500744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:44.501584 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:44.501916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:44.502199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:44.504185 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:44.516810 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:44.518460 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:44.530023 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1373/2760] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:44.556589 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c53bf1554a4825afd8770ea9347ad1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.557257 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c53bf1554a4825afd8770ea9347ad1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.557906 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c53bf1554a4825afd8770ea9347ad1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.566904 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c53bf1554a4825afd8770ea9347ad1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.575756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-68b6c484-2c08-4b6b-abf3-90d5f62f8923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c53bf1554a4825afd8770ea9347ad1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.595316 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1388/2761] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:29:44 2022] DELETE /identity/v3/users/c6c53bf1554a4825afd8770ea9347ad1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:29:44.798481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:44.811651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160e099-154c-49c8-a3b3-1bc25bf7dc87 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:44.874859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:44.875721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:44.876021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:44.876281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] PATH_INFO: `/v3/projects/41f6c4a1b7784d7d874b00d5e43507cf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:44.877455 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f6c4a1b7784d7d874b00d5e43507cf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:44.879513 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:44.897277 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:44.928977 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f6c4a1b7784d7d874b00d5e43507cf', 'request_id': 'req-5d368c5a-d72f-4bb2-aa48-d85018c769d5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.937817 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f6c4a1b7784d7d874b00d5e43507cf', 'request_id': 'req-5d368c5a-d72f-4bb2-aa48-d85018c769d5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.944169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f6c4a1b7784d7d874b00d5e43507cf', 'request_id': 'req-5d368c5a-d72f-4bb2-aa48-d85018c769d5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.957585 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] The token cache is being invalidate because project 41f6c4a1b7784d7d874b00d5e43507cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:29:44.957918 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.958574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d368c5a-d72f-4bb2-aa48-d85018c769d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:44.960123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1374/2762] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:29:44 2022] DELETE /identity/v3/projects/41f6c4a1b7784d7d874b00d5e43507cf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:44.965764 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf93c8b-2af6-4e5f-ba30-874746284945 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:44.966085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf93c8b-2af6-4e5f-ba30-874746284945 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:44.966367 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf93c8b-2af6-4e5f-ba30-874746284945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:44.984914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4bf93c8b-2af6-4e5f-ba30-874746284945 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:44.986150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4bf93c8b-2af6-4e5f-ba30-874746284945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:45.050779 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1389/2763] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:45.059206 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0ee18-e609-42e6-825b-df1759fffecf None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.059498 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0ee18-e609-42e6-825b-df1759fffecf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.059781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0ee18-e609-42e6-825b-df1759fffecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.081092 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9fa0ee18-e609-42e6-825b-df1759fffecf None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:45.082820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9fa0ee18-e609-42e6-825b-df1759fffecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:45.091864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1375/2764] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:45.096400 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf93c8b-2af6-4e5f-ba30-874746284945 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.110622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf93c8b-2af6-4e5f-ba30-874746284945 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.118222 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.119223 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.119654 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.120373 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.121208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:45.121685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.124324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.126724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:45.144930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1390/2765] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:45.149982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0ee18-e609-42e6-825b-df1759fffecf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.159418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0ee18-e609-42e6-825b-df1759fffecf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.164696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.165413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.165799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.166281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.167288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.169278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.199377 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1376/2766] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:29:45 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:45.203387 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.214138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49080537-a5a2-42c7-a509-c187a3b9a40b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.275854 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.276576 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.276864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.277144 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.277895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.279845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.301280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1391/2767] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:29:45 2022] POST /identity/v3/users => generated 281 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:45.305848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.316572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380ef04-0707-4fe4-ac06-268ede07e9de admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.321320 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.322038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.322328 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.322610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.323334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.324943 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.336987 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1377/2768] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:45.340989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.352213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd180b-b9c0-4839-93f3-05b5fdbcb568 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.356910 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bcb05-2ae2-4493-a855-191b0442f4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.357585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bcb05-2ae2-4493-a855-191b0442f4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.357908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bcb05-2ae2-4493-a855-191b0442f4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.358189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bcb05-2ae2-4493-a855-191b0442f4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.358917 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bcb05-2ae2-4493-a855-191b0442f4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.360568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bcb05-2ae2-4493-a855-191b0442f4b5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.371612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1392/2769] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:45.375776 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.388769 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25203d25-2cbb-4cb6-a3ea-24e016e9aae9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.393424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.394391 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.394861 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.394973 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] PATH_INFO: `/v3/projects/a29a0b7dbd9640beba1f166a1dc7e946/users/582bd8bd7668426aa0b944013c1eba06/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.411636 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29a0b7dbd9640beba1f166a1dc7e946, user_id=582bd8bd7668426aa0b944013c1eba06, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.418095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.434972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1378/2770] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:29:45 2022] PUT /identity/v3/projects/a29a0b7dbd9640beba1f166a1dc7e946/users/582bd8bd7668426aa0b944013c1eba06/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:45.445882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae70b4-ba5e-42f4-826e-de6f75cb6896 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.446250 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae70b4-ba5e-42f4-826e-de6f75cb6896 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.446532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae70b4-ba5e-42f4-826e-de6f75cb6896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.477407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0aae70b4-ba5e-42f4-826e-de6f75cb6896 None None] MFA Rules not processed for user `582bd8bd7668426aa0b944013c1eba06`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:45.478711 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0aae70b4-ba5e-42f4-826e-de6f75cb6896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:45.545444 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1393/2771] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:45.550345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.618181 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e3662-e198-4c89-b25e-e81ecc87b5c5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.622741 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.623407 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.623691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.623969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.624655 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.626606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.658093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1379/2772] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:29:45 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:45.659170 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae70b4-ba5e-42f4-826e-de6f75cb6896 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.671857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae70b4-ba5e-42f4-826e-de6f75cb6896 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.731189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.731885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.732176 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.732477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.733169 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.735138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.743519 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:45.764563 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1394/2773] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:29:45 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:29:45.765945 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.780290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d08acd-2c3c-487a-9804-fb479a091659 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.787586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.788601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.789108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.789542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.790583 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.792367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.804258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1380/2774] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:45.808635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.820708 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909f2e2-9ce4-46a1-861d-c578cb8f395b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.827579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.829030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.829491 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.829981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] PATH_INFO: `/v3/projects/3a9f317261454c6a8af9a6234a06949c/users/6bb5233513ed4524a8898e91fc35a63f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.848705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9f317261454c6a8af9a6234a06949c, user_id=6bb5233513ed4524a8898e91fc35a63f, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.851476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.866701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1395/2775] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:29:45 2022] PUT /identity/v3/projects/3a9f317261454c6a8af9a6234a06949c/users/6bb5233513ed4524a8898e91fc35a63f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:45.870917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.881100 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d6add-38ed-43c8-a4c1-00065b247ab6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.897932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:45.953372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083a805-3b36-46f5-b388-e951f2940c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.954088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a805-3b36-46f5-b388-e951f2940c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.954378 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a805-3b36-46f5-b388-e951f2940c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.954683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a805-3b36-46f5-b388-e951f2940c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.955463 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083a805-3b36-46f5-b388-e951f2940c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.957113 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083a805-3b36-46f5-b388-e951f2940c97 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:45.968816 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1381/2776] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:45.973765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:45.984756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b320a39-17db-430b-8678-a30b6aa4a89a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:45.990195 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:45.990545 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:45.990829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:45.991160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] PATH_INFO: `/v3/projects/3a9f317261454c6a8af9a6234a06949c/users/6bb5233513ed4524a8898e91fc35a63f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:45.994037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9f317261454c6a8af9a6234a06949c, user_id=6bb5233513ed4524a8898e91fc35a63f, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:45.996701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:46.008555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1396/2777] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:29:45 2022] PUT /identity/v3/projects/3a9f317261454c6a8af9a6234a06949c/users/6bb5233513ed4524a8898e91fc35a63f/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:46.018860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8846ee-5986-4578-9d93-c84b53c9e1ff None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:46.019151 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8846ee-5986-4578-9d93-c84b53c9e1ff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:46.019453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8846ee-5986-4578-9d93-c84b53c9e1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:46.047064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2a8846ee-5986-4578-9d93-c84b53c9e1ff None None] MFA Rules not processed for user `6bb5233513ed4524a8898e91fc35a63f`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:46.048288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2a8846ee-5986-4578-9d93-c84b53c9e1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:46.122199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1382/2778] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:46.135081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:46.140052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:46.203034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2255d4-56bf-48f1-8561-9ed3168b1450 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:46.207445 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:46.208248 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:46.208534 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:46.208843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:46.219549 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:46.221007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:46.233247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1397/2779] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:46 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:46.315469 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8846ee-5986-4578-9d93-c84b53c9e1ff None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:46.326883 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8846ee-5986-4578-9d93-c84b53c9e1ff None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:46.330475 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf072f6f-3ace-491a-9196-677299c6cc01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:46.331167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf072f6f-3ace-491a-9196-677299c6cc01 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:46.331413 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf072f6f-3ace-491a-9196-677299c6cc01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:46.331690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf072f6f-3ace-491a-9196-677299c6cc01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:46.393247 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf072f6f-3ace-491a-9196-677299c6cc01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:46.394647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf072f6f-3ace-491a-9196-677299c6cc01 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:46.404682 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1383/2780] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:49.152202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:49.219868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972bf11-365d-4d88-a7c9-68dd1641a7fd service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:49.225595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:49.226538 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:49.226828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:49.227114 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] PATH_INFO: `/v3/users/582bd8bd7668426aa0b944013c1eba06` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:49.228380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] RBAC: Authorizing `identity:delete_user(user_id=582bd8bd7668426aa0b944013c1eba06)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:49.230376 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:49.289954 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582bd8bd7668426aa0b944013c1eba06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.290530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582bd8bd7668426aa0b944013c1eba06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.291137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582bd8bd7668426aa0b944013c1eba06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.298577 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582bd8bd7668426aa0b944013c1eba06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.305368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:49.309254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582bd8bd7668426aa0b944013c1eba06'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.331635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1398/2781] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:29:49 2022] DELETE /identity/v3/users/582bd8bd7668426aa0b944013c1eba06 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:49.336800 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:49.337321 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:49.337431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:49.355740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:49.356967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:49.390789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:49.421237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1384/2782] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:29:49.431838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:49.433306 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:49.493034 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabefb24-dcc8-45fc-b926-007eb2e7c43c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:49.496685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:49.497342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:49.497629 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:49.497943 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:49.507483 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:49.508934 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:49.518834 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1399/2783] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:49.816282 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:49.818362 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:49.830743 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1261c-d010-4a6a-8933-a46ab3f17e21 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:49.836189 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:49.837082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:49.837367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:49.837645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] PATH_INFO: `/v3/projects/a29a0b7dbd9640beba1f166a1dc7e946` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:49.838897 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29a0b7dbd9640beba1f166a1dc7e946)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:49.840833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:49.900476 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29a0b7dbd9640beba1f166a1dc7e946', 'request_id': 'req-11d5dfa3-ff35-483e-b29c-3e0743ad0205'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.908272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29a0b7dbd9640beba1f166a1dc7e946', 'request_id': 'req-11d5dfa3-ff35-483e-b29c-3e0743ad0205'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.915350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29a0b7dbd9640beba1f166a1dc7e946', 'request_id': 'req-11d5dfa3-ff35-483e-b29c-3e0743ad0205'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.926000 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] The token cache is being invalidate because project a29a0b7dbd9640beba1f166a1dc7e946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:29:49.926288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.926969 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:49.928712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1385/2784] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:29:49 2022] DELETE /identity/v3/projects/a29a0b7dbd9640beba1f166a1dc7e946 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:49.932762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:49.943571 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708da1a9-ff3a-4114-8ad3-44c375eab55a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:50.008022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:50.009322 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.009604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.009924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] PATH_INFO: `/v3/users/6bb5233513ed4524a8898e91fc35a63f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.011118 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb5233513ed4524a8898e91fc35a63f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:50.013052 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:50.060967 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb5233513ed4524a8898e91fc35a63f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.061637 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb5233513ed4524a8898e91fc35a63f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.062263 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb5233513ed4524a8898e91fc35a63f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.071259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb5233513ed4524a8898e91fc35a63f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.078697 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-48fe3eb1-f144-40d7-9150-31ee3c4a65b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb5233513ed4524a8898e91fc35a63f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.092692 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:50.101389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1400/2785] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:29:49 2022] DELETE /identity/v3/users/6bb5233513ed4524a8898e91fc35a63f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:50.400068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:50.401596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:50.419864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d5dfa3-ff35-483e-b29c-3e0743ad0205 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:50.483150 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:50.484055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.484336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.484610 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] PATH_INFO: `/v3/projects/3a9f317261454c6a8af9a6234a06949c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.485996 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9f317261454c6a8af9a6234a06949c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:50.487979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:50.541479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9f317261454c6a8af9a6234a06949c', 'request_id': 'req-675acb06-3c6a-4d34-97f3-6b0b974a570b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.549001 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9f317261454c6a8af9a6234a06949c', 'request_id': 'req-675acb06-3c6a-4d34-97f3-6b0b974a570b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.559176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9f317261454c6a8af9a6234a06949c', 'request_id': 'req-675acb06-3c6a-4d34-97f3-6b0b974a570b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.560896 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:50.575857 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] The token cache is being invalidate because project 3a9f317261454c6a8af9a6234a06949c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:29:50.576146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.576844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-675acb06-3c6a-4d34-97f3-6b0b974a570b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:50.578860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1386/2786] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:29:50 2022] DELETE /identity/v3/projects/3a9f317261454c6a8af9a6234a06949c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:50.583838 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f894-b729-4287-b41b-7f178683359d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.584324 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f894-b729-4287-b41b-7f178683359d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.584670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f894-b729-4287-b41b-7f178683359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.603112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9f05f894-b729-4287-b41b-7f178683359d None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:50.604340 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f05f894-b729-4287-b41b-7f178683359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:50.677179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1401/2787] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:50.685586 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf77a-cb5c-497e-8819-66f3ebe36714 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.685910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf77a-cb5c-497e-8819-66f3ebe36714 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.686199 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffcf77a-cb5c-497e-8819-66f3ebe36714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.706267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bffcf77a-cb5c-497e-8819-66f3ebe36714 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:50.707484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bffcf77a-cb5c-497e-8819-66f3ebe36714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:50.716598 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1387/2788] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 16:29:50.721138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05f894-b729-4287-b41b-7f178683359d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:50.731579 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05f894-b729-4287-b41b-7f178683359d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:50.736672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:50.737369 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.737658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.737973 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.738715 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:50.739031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:50.740665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:50.754029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1402/2789] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:50.759149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcf77a-cb5c-497e-8819-66f3ebe36714 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:50.768922 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffcf77a-cb5c-497e-8819-66f3ebe36714 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:50.773859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:50.775555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.775555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.775555 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.776059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:50.777746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:50.808306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1388/2790] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:29:50 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:29:50.813097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:50.825377 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5c023-9b34-44e6-8e3e-400d4d4d07f4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:50.892876 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:50.893695 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.894392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.894738 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.895605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:50.897593 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:50.919030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1403/2791] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:29:50 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:50.923164 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:50.935373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b41cad-fe10-426c-99b7-a9f53b97a16e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:50.939959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:50.940654 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.940941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.941219 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.941975 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:50.943572 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:50.955419 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1389/2792] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:50.960081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:50.970682 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bff90-4944-4a53-aa2b-113dd730d20a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:50.975681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c3ce8-6c98-4d40-86fd-d6e5eef70e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:50.977014 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c3ce8-6c98-4d40-86fd-d6e5eef70e66 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:50.977303 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c3ce8-6c98-4d40-86fd-d6e5eef70e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:50.977578 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c3ce8-6c98-4d40-86fd-d6e5eef70e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:50.978356 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c3ce8-6c98-4d40-86fd-d6e5eef70e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:50.979991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c3ce8-6c98-4d40-86fd-d6e5eef70e66 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:50.995018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1404/2793] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:50.999431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.010606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7be3ab-bced-480f-a89b-6aee40bdb7d8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.015536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.016431 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.016716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.016999 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] PATH_INFO: `/v3/projects/98cd7090f8f94308886dfd3a3985a676/users/feabc5b2c260452594a8c2700b217bd7/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.033082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cd7090f8f94308886dfd3a3985a676, user_id=feabc5b2c260452594a8c2700b217bd7, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.035801 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.049985 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1390/2794] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:29:50 2022] PUT /identity/v3/projects/98cd7090f8f94308886dfd3a3985a676/users/feabc5b2c260452594a8c2700b217bd7/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:51.063885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f358c-3777-4021-ba63-da672d43da1d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.064031 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f358c-3777-4021-ba63-da672d43da1d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.064497 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f358c-3777-4021-ba63-da672d43da1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.092686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-992f358c-3777-4021-ba63-da672d43da1d None None] MFA Rules not processed for user `feabc5b2c260452594a8c2700b217bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:51.093959 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-992f358c-3777-4021-ba63-da672d43da1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:51.156389 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:51.170408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1405/2795] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:51.175018 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.239314 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eec86-adef-4ee4-9e29-8271fb30c6ce admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.244392 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.245064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.245341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.245616 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.246380 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.248319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.276559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1391/2796] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:29:51 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:51.279183 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f358c-3777-4021-ba63-da672d43da1d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.291322 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f358c-3777-4021-ba63-da672d43da1d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.352615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.353293 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.353583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.353905 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.354617 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.356558 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.378569 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1406/2797] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:29:51 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:51.382146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.392892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397fdc92-9312-4197-819f-93d8d6ec051f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.400753 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.401425 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.401712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.402027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.402755 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.404354 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.417902 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1392/2798] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:51.422401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.432684 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2cbb9-7987-4450-b9b7-973950a2654a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.437290 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.438261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.438547 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.438824 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] PATH_INFO: `/v3/projects/4f9d890627ae4a4d8e9390234c2087fc/users/df7bdc67d7fa4580bfa4cb51aaef411c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.454612 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9d890627ae4a4d8e9390234c2087fc, user_id=df7bdc67d7fa4580bfa4cb51aaef411c, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.457379 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.477411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1407/2799] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:29:51 2022] PUT /identity/v3/projects/4f9d890627ae4a4d8e9390234c2087fc/users/df7bdc67d7fa4580bfa4cb51aaef411c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:51.481754 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.492332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad277f-f1b0-4afd-b626-0ab106a8472b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.555021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deeb483-7b3d-4ea3-9171-15e64a7de525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.555696 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeb483-7b3d-4ea3-9171-15e64a7de525 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.555980 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeb483-7b3d-4ea3-9171-15e64a7de525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.556258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deeb483-7b3d-4ea3-9171-15e64a7de525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.557154 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deeb483-7b3d-4ea3-9171-15e64a7de525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.558788 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deeb483-7b3d-4ea3-9171-15e64a7de525 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.560264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:51.576070 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1393/2800] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:51.580259 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.591663 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328ef66-9626-4fea-a0d0-b8b0e43a3c59 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.596358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.597247 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.597535 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.597851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] PATH_INFO: `/v3/projects/4f9d890627ae4a4d8e9390234c2087fc/users/df7bdc67d7fa4580bfa4cb51aaef411c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.600139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9d890627ae4a4d8e9390234c2087fc, user_id=df7bdc67d7fa4580bfa4cb51aaef411c, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.602941 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.613915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1408/2801] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:29:51 2022] PUT /identity/v3/projects/4f9d890627ae4a4d8e9390234c2087fc/users/df7bdc67d7fa4580bfa4cb51aaef411c/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:51.627027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e2a8-e498-40a4-a34e-aa98bff1dc19 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.627316 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e2a8-e498-40a4-a34e-aa98bff1dc19 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.627595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e2a8-e498-40a4-a34e-aa98bff1dc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.658234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b201e2a8-e498-40a4-a34e-aa98bff1dc19 None None] MFA Rules not processed for user `df7bdc67d7fa4580bfa4cb51aaef411c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:51.659459 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b201e2a8-e498-40a4-a34e-aa98bff1dc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:51.734090 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1394/2802] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:51.745364 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.747160 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:51.816833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc02fc-d515-4ee4-afef-7b8a0b112e3f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.821015 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f7fa1-5a5c-45b6-a59c-6b0414873560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.821670 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7fa1-5a5c-45b6-a59c-6b0414873560 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.821991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7fa1-5a5c-45b6-a59c-6b0414873560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.822272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7fa1-5a5c-45b6-a59c-6b0414873560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.833997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f7fa1-5a5c-45b6-a59c-6b0414873560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.835368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f7fa1-5a5c-45b6-a59c-6b0414873560 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:51.846238 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1409/2803] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:51.876279 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e2a8-e498-40a4-a34e-aa98bff1dc19 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:51.948952 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e2a8-e498-40a4-a34e-aa98bff1dc19 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:51.953601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:51.954588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:51.955062 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:51.955335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] PATH_INFO: `/v3/users/feabc5b2c260452594a8c2700b217bd7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:51.956569 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=feabc5b2c260452594a8c2700b217bd7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:51.958532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:52.008652 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feabc5b2c260452594a8c2700b217bd7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.009311 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feabc5b2c260452594a8c2700b217bd7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.017411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feabc5b2c260452594a8c2700b217bd7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.018507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feabc5b2c260452594a8c2700b217bd7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.028664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feabc5b2c260452594a8c2700b217bd7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.043532 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:52.048115 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1395/2804] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:29:51 2022] DELETE /identity/v3/users/feabc5b2c260452594a8c2700b217bd7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:52.054111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:52.055243 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:52.055742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:52.076975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:52.078789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:52.148907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:52.165895 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1410/2805] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:52.177353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:52.179174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:52.252510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c4fbd7-9877-4703-955e-bb11198652fd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:52.256402 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:52.257056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:52.257342 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:52.258405 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:52.268013 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:52.270021 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:52.280241 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1396/2806] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:29:52.565983 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:52.567455 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:52.582720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e5f50-8f3c-4ec6-90a8-a267a0e4560e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:52.587962 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:52.588841 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:52.589121 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:52.590272 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] PATH_INFO: `/v3/projects/98cd7090f8f94308886dfd3a3985a676` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:52.592192 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=98cd7090f8f94308886dfd3a3985a676)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:52.594157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:52.644680 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98cd7090f8f94308886dfd3a3985a676', 'request_id': 'req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.654713 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98cd7090f8f94308886dfd3a3985a676', 'request_id': 'req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.661319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98cd7090f8f94308886dfd3a3985a676', 'request_id': 'req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.672161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] The token cache is being invalidate because project 98cd7090f8f94308886dfd3a3985a676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:29:52.672452 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.673211 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.674935 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1411/2807] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:29:52 2022] DELETE /identity/v3/projects/98cd7090f8f94308886dfd3a3985a676 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:29:52.679959 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:52.689935 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dababb-56fb-48df-9cec-68a5f33783b5 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:52.753855 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:52.754766 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:52.755057 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:52.755341 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] PATH_INFO: `/v3/users/df7bdc67d7fa4580bfa4cb51aaef411c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:52.756565 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7bdc67d7fa4580bfa4cb51aaef411c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:52.759127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:52.814467 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7bdc67d7fa4580bfa4cb51aaef411c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.815445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7bdc67d7fa4580bfa4cb51aaef411c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.826609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7bdc67d7fa4580bfa4cb51aaef411c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.827336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7bdc67d7fa4580bfa4cb51aaef411c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.836424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b60d748b-c766-425b-b417-a2885c3a232a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7bdc67d7fa4580bfa4cb51aaef411c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:52.855647 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1397/2808] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:29:52 2022] DELETE /identity/v3/users/df7bdc67d7fa4580bfa4cb51aaef411c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:53.156947 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.158421 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:53.171862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2ed1b-e351-4045-901b-b2b1d7c34b8c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.234482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.235350 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.235635 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.235919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] PATH_INFO: `/v3/projects/4f9d890627ae4a4d8e9390234c2087fc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.237140 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9d890627ae4a4d8e9390234c2087fc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.239111 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.288028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9d890627ae4a4d8e9390234c2087fc', 'request_id': 'req-2af57485-48ad-4238-8358-dd21fc1ca0ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:53.295859 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9d890627ae4a4d8e9390234c2087fc', 'request_id': 'req-2af57485-48ad-4238-8358-dd21fc1ca0ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:53.302721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9d890627ae4a4d8e9390234c2087fc', 'request_id': 'req-2af57485-48ad-4238-8358-dd21fc1ca0ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:53.323115 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] The token cache is being invalidate because project 4f9d890627ae4a4d8e9390234c2087fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:29:53.323343 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:53.323986 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2af57485-48ad-4238-8358-dd21fc1ca0ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:29:53.325873 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1412/2809] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:29:53 2022] DELETE /identity/v3/projects/4f9d890627ae4a4d8e9390234c2087fc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:53.330955 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc53fc-6ef2-491e-893e-16123bf71150 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.331305 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc53fc-6ef2-491e-893e-16123bf71150 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.331507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc53fc-6ef2-491e-893e-16123bf71150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.350970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e6cc53fc-6ef2-491e-893e-16123bf71150 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:53.352174 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e6cc53fc-6ef2-491e-893e-16123bf71150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:53.371433 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:53.416068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1398/2810] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:53.423744 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42724a-1911-4202-886c-ff365b908bda None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.424045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42724a-1911-4202-886c-ff365b908bda None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.424325 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42724a-1911-4202-886c-ff365b908bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.431349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:29:53.445929 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a42724a-1911-4202-886c-ff365b908bda None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:53.447130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a42724a-1911-4202-886c-ff365b908bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:53.455414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1413/2811] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:53.459690 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc53fc-6ef2-491e-893e-16123bf71150 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.469353 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc53fc-6ef2-491e-893e-16123bf71150 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.475256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.476325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.476772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.477188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.478332 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:29:53.478774 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.481260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.494233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1399/2812] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:29:53.498611 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42724a-1911-4202-886c-ff365b908bda None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.508855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42724a-1911-4202-886c-ff365b908bda None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.513512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.514208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.514529 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.514810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.515515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.517430 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.547559 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1414/2813] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:29:53 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:29:53.551956 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.562510 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56100b79-86bc-4ace-97eb-e1694fa67144 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.621003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.621596 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.621917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.622197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.622895 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.624825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.647659 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1400/2814] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:29:53 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:29:53.651843 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.662252 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea56cf-21c5-4d8d-80ca-f674285efaf3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.666822 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.667479 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.667765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.668041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.668760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.670402 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.683521 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1415/2815] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:53.685627 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.696542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba66fe-9a5f-4961-95ae-9b697bc31434 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.701370 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995480c-0989-41ac-a7e3-0a6cf8bc7d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.702056 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995480c-0989-41ac-a7e3-0a6cf8bc7d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.702347 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995480c-0989-41ac-a7e3-0a6cf8bc7d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.702624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995480c-0989-41ac-a7e3-0a6cf8bc7d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.703363 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995480c-0989-41ac-a7e3-0a6cf8bc7d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.704953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995480c-0989-41ac-a7e3-0a6cf8bc7d16 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.716972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1401/2816] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:29:53.721462 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.731849 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85877f-3b72-4592-bc2c-47b9a818b45a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.736454 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.737985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.737985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.737985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] PATH_INFO: `/v3/projects/48d467e06b6947798e00e98a91bd4733/users/856c878e18744d16bb44b6a71216ef6e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.753213 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d467e06b6947798e00e98a91bd4733, user_id=856c878e18744d16bb44b6a71216ef6e, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.755900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.769799 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1416/2817] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:29:53 2022] PUT /identity/v3/projects/48d467e06b6947798e00e98a91bd4733/users/856c878e18744d16bb44b6a71216ef6e/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:29:53.780900 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f946b2-6dd5-4151-bd98-b99a04b59511 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.781220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f946b2-6dd5-4151-bd98-b99a04b59511 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.781501 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f946b2-6dd5-4151-bd98-b99a04b59511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.808552 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-22f946b2-6dd5-4151-bd98-b99a04b59511 None None] MFA Rules not processed for user `856c878e18744d16bb44b6a71216ef6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:29:53.810260 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-22f946b2-6dd5-4151-bd98-b99a04b59511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:53.878242 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1402/2818] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:29:53.889317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:29:53.891202 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:29:53.952221 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70529f0-454c-4951-89de-8d8a0ea17857 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:29:53.955868 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e3343-6607-462e-b86a-609477367560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:29:53.956516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e3343-6607-462e-b86a-609477367560 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:29:53.956802 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e3343-6607-462e-b86a-609477367560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:29:53.957082 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e3343-6607-462e-b86a-609477367560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:29:53.965650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e3343-6607-462e-b86a-609477367560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:29:53.967122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e3343-6607-462e-b86a-609477367560 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:29:53.977010 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1417/2819] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:29:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:30:03.875325 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:03.972155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:17.864290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f946b2-6dd5-4151-bd98-b99a04b59511 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:17.926225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f946b2-6dd5-4151-bd98-b99a04b59511 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:17.930718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:17.931580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:17.931862 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:17.932140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] PATH_INFO: `/v3/users/856c878e18744d16bb44b6a71216ef6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:17.933307 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] RBAC: Authorizing `identity:delete_user(user_id=856c878e18744d16bb44b6a71216ef6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:17.935263 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:17.980626 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '856c878e18744d16bb44b6a71216ef6e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:17.981300 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '856c878e18744d16bb44b6a71216ef6e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:17.988096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '856c878e18744d16bb44b6a71216ef6e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:17.988718 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '856c878e18744d16bb44b6a71216ef6e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:17.996028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '856c878e18744d16bb44b6a71216ef6e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:18.013833 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1403/2820] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:30:17 2022] DELETE /identity/v3/users/856c878e18744d16bb44b6a71216ef6e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:18.018326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.018794 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.018927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.036200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:18.037489 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:18.097249 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1418/2821] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:18.107795 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.109191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:18.168237 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8429d11-0b11-4dea-a9c6-68329154969e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.171970 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b244180-d01d-40f2-bc0c-afc1713eb689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.172612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b244180-d01d-40f2-bc0c-afc1713eb689 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.172901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b244180-d01d-40f2-bc0c-afc1713eb689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.173178 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b244180-d01d-40f2-bc0c-afc1713eb689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.182606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b244180-d01d-40f2-bc0c-afc1713eb689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:18.184027 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b244180-d01d-40f2-bc0c-afc1713eb689 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:18.193998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1404/2822] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:18.468737 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.470375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:18.486533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a6468-58a7-4f61-8d3a-ab924463cb9e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.492501 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.493353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.493707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.493948 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] PATH_INFO: `/v3/projects/48d467e06b6947798e00e98a91bd4733` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.495177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d467e06b6947798e00e98a91bd4733)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:18.497108 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:18.548457 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d467e06b6947798e00e98a91bd4733', 'request_id': 'req-c8eeff35-6f49-455e-be5d-d2f6872c26d5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:18.560598 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d467e06b6947798e00e98a91bd4733', 'request_id': 'req-c8eeff35-6f49-455e-be5d-d2f6872c26d5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:18.567215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d467e06b6947798e00e98a91bd4733', 'request_id': 'req-c8eeff35-6f49-455e-be5d-d2f6872c26d5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:18.578093 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] The token cache is being invalidate because project 48d467e06b6947798e00e98a91bd4733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:30:18.578408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:18.579039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c8eeff35-6f49-455e-be5d-d2f6872c26d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:18.580632 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1419/2823] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:30:18 2022] DELETE /identity/v3/projects/48d467e06b6947798e00e98a91bd4733 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:18.586231 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cec751-1b31-4642-b7fc-a28680e4873c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.586520 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cec751-1b31-4642-b7fc-a28680e4873c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.586811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cec751-1b31-4642-b7fc-a28680e4873c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.604961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c2cec751-1b31-4642-b7fc-a28680e4873c None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:18.606225 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2cec751-1b31-4642-b7fc-a28680e4873c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:18.623763 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:18.670108 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1405/2824] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:18.677909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a332-a437-4c8f-bc11-ccc81d5b9870 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.678286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a332-a437-4c8f-bc11-ccc81d5b9870 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.678575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a332-a437-4c8f-bc11-ccc81d5b9870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.697304 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-73a0a332-a437-4c8f-bc11-ccc81d5b9870 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:18.698537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-73a0a332-a437-4c8f-bc11-ccc81d5b9870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:18.706407 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1420/2825] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:18.710726 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cec751-1b31-4642-b7fc-a28680e4873c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.720702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cec751-1b31-4642-b7fc-a28680e4873c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.725486 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.726197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.726479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.726756 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.727464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:18.727768 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:18.729346 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:18.741037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1406/2826] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:18.744969 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0a332-a437-4c8f-bc11-ccc81d5b9870 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.754939 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0a332-a437-4c8f-bc11-ccc81d5b9870 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.760234 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.760902 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.761189 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.761471 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.762276 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:18.764194 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:18.791076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1421/2827] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:30:18 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:18.796432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.807048 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d61ae-6f9b-4a20-9eda-bc0fd56b709a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.870145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.870836 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.871084 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.871360 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.872038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:18.873964 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:18.894653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1407/2828] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:30:18 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:18.899726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.910181 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3085e1-1931-4cb9-98ef-15f24abf055a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.914763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.915433 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.915725 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.916006 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.916741 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:18.918423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:18.930742 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1422/2829] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:18.934781 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.946892 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fcb6c-cd0e-4ab8-a0c7-fd703bf1fbe9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.952180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0c9e-35b9-4b01-a0c9-b1175dbb9474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.952845 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0c9e-35b9-4b01-a0c9-b1175dbb9474 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.953135 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0c9e-35b9-4b01-a0c9-b1175dbb9474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.953842 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0c9e-35b9-4b01-a0c9-b1175dbb9474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:18.954576 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc0c9e-35b9-4b01-a0c9-b1175dbb9474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:18.956208 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc0c9e-35b9-4b01-a0c9-b1175dbb9474 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:18.968254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1408/2830] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:30:18.972564 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:18.983423 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c3e75-0c3e-485d-b5d5-4ee2ba0f6da5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:18.988477 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:18.989408 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:18.989705 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:18.990045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] PATH_INFO: `/v3/projects/ac679b3abab440f3a9adf0bffba6a7b3/users/12c0485871114a0088ac6920bbbc3dbc/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.005182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac679b3abab440f3a9adf0bffba6a7b3, user_id=12c0485871114a0088ac6920bbbc3dbc, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.008796 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.021976 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1423/2831] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:18 2022] PUT /identity/v3/projects/ac679b3abab440f3a9adf0bffba6a7b3/users/12c0485871114a0088ac6920bbbc3dbc/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:19.033271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d87a-77e6-47b0-aad3-929b445afccd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.033643 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d87a-77e6-47b0-aad3-929b445afccd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.033946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b665d87a-77e6-47b0-aad3-929b445afccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.061941 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b665d87a-77e6-47b0-aad3-929b445afccd None None] MFA Rules not processed for user `12c0485871114a0088ac6920bbbc3dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:19.063588 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b665d87a-77e6-47b0-aad3-929b445afccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:19.132775 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1409/2832] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:19.143358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.166039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:19.212512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c49aae-3db3-4749-8475-e8bdd20ecccf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.217302 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.218012 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.218301 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.218582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.219295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.221235 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.254329 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1424/2833] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:30:19 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:19.255504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665d87a-77e6-47b0-aad3-929b445afccd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.266998 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b665d87a-77e6-47b0-aad3-929b445afccd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.281504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:19.326877 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.327471 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.327773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.328045 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.328746 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.330699 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.351077 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1410/2834] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:30:19 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:30:19.355201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.365347 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404779-a417-473d-87aa-b31e0bc28fef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.370076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.370739 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.371026 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.371308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.372037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.373644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.384814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1425/2835] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:30:19.388867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.399559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf775e-f0e1-47c7-b6da-2ec0e03b9ea8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.404118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9afcc6-f884-4e07-ae4e-c81898810319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.404773 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9afcc6-f884-4e07-ae4e-c81898810319 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.405061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9afcc6-f884-4e07-ae4e-c81898810319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.405333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9afcc6-f884-4e07-ae4e-c81898810319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.406202 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9afcc6-f884-4e07-ae4e-c81898810319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.407794 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9afcc6-f884-4e07-ae4e-c81898810319 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.417792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1411/2836] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:19.421961 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.431963 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951ea92-463d-4b5b-a855-13add3f70574 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.436585 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.437530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.437769 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.438055 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] PATH_INFO: `/v3/projects/644d9f4c8f744ff2aeb300795c0d65ce/users/5fb5ffdb4bc24e04b3976fd3b0612288/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.453165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] RBAC: Authorizing `identity:create_grant(project_id=644d9f4c8f744ff2aeb300795c0d65ce, user_id=5fb5ffdb4bc24e04b3976fd3b0612288, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.455855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.468996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1426/2837] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:19 2022] PUT /identity/v3/projects/644d9f4c8f744ff2aeb300795c0d65ce/users/5fb5ffdb4bc24e04b3976fd3b0612288/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:19.479491 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffd6ee-4533-402d-b21f-6d1605413c0f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.479992 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffd6ee-4533-402d-b21f-6d1605413c0f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.480276 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffd6ee-4533-402d-b21f-6d1605413c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.507445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-baffd6ee-4533-402d-b21f-6d1605413c0f None None] MFA Rules not processed for user `5fb5ffdb4bc24e04b3976fd3b0612288`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:19.508761 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-baffd6ee-4533-402d-b21f-6d1605413c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:19.578117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1412/2838] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:19.582938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.648702 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4fef1-999a-4424-a967-c7002579164f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.654997 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.655701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.656030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.656618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.657308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.659426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.686888 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1427/2839] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:30:19 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:19.691545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffd6ee-4533-402d-b21f-6d1605413c0f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.708783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffd6ee-4533-402d-b21f-6d1605413c0f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.774087 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.774767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.775054 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.775336 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.776037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.778020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.797901 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1413/2840] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:30:19 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:30:19.802041 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.813231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be7190-06d2-4dfb-a8e8-90e26f564bc9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.817977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.818642 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.818930 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.819240 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.820596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.822266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.833011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1428/2841] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:19.837205 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.847256 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced77755-5def-4bbc-8504-aa66c5b69c87 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.852075 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.852953 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.853234 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.853507 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] PATH_INFO: `/v3/projects/235c531ded114f1cbb588408813884a0/users/642cbd61016947ffa25043b214205269/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.865274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:19.869032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=235c531ded114f1cbb588408813884a0, user_id=642cbd61016947ffa25043b214205269, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.872144 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.885907 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1414/2842] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:19 2022] PUT /identity/v3/projects/235c531ded114f1cbb588408813884a0/users/642cbd61016947ffa25043b214205269/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:19.890016 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.901979 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe6132-169c-4e91-aaac-2d1c5d80355f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:19.945061 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:19.963927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6f1ff-2bf4-46c7-8d26-08bf154cf6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:19.965179 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f1ff-2bf4-46c7-8d26-08bf154cf6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:19.965513 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f1ff-2bf4-46c7-8d26-08bf154cf6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:19.965842 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6f1ff-2bf4-46c7-8d26-08bf154cf6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:19.966626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f1ff-2bf4-46c7-8d26-08bf154cf6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:19.968390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6f1ff-2bf4-46c7-8d26-08bf154cf6f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:19.980076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1429/2843] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:19.984535 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:19.995656 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283eb54-7e19-42bd-a698-33a20b79e63f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:20.000309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:20.001188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:20.001474 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:20.001783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] PATH_INFO: `/v3/projects/235c531ded114f1cbb588408813884a0/users/642cbd61016947ffa25043b214205269/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:20.003826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] RBAC: Authorizing `identity:create_grant(project_id=235c531ded114f1cbb588408813884a0, user_id=642cbd61016947ffa25043b214205269, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:20.006497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:20.017372 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1415/2844] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:19 2022] PUT /identity/v3/projects/235c531ded114f1cbb588408813884a0/users/642cbd61016947ffa25043b214205269/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:20.027848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:20.028391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:20.028391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:20.055658 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] MFA Rules not processed for user `642cbd61016947ffa25043b214205269`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:20.056884 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:20.132353 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1430/2845] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:20.143849 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:20.145697 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:20.203038 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cae0bc-e67b-49fe-894f-54cc22a3425c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:20.207008 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:20.207716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:20.208005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:20.208288 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:20.264967 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:20.266518 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:20.275678 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1416/2846] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:21.770476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:21.772606 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:21.792231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd1eb9-4fbb-40df-a9fe-bdb03a4abf00 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:21.797768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:21.798806 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:21.799286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:21.799756 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:21.868786 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:21.870212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:21.879182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1431/2847] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:22.438140 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:22.447707 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a05b-59e8-4930-8546-ae41231f4bc8 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:22.451441 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f98e1-bbf3-471d-aaeb-2f02adcd34d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:22.452096 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f98e1-bbf3-471d-aaeb-2f02adcd34d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:22.452383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f98e1-bbf3-471d-aaeb-2f02adcd34d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:22.452674 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f98e1-bbf3-471d-aaeb-2f02adcd34d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:22.462995 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f98e1-bbf3-471d-aaeb-2f02adcd34d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:22.464437 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f98e1-bbf3-471d-aaeb-2f02adcd34d3 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:22.475424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1417/2848] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:26.363928 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:26.429375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe5f76-54a8-43de-acc3-3ca2cba17013 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:26.433898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:26.434772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:26.435051 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:26.435323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] PATH_INFO: `/v3/users/12c0485871114a0088ac6920bbbc3dbc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:26.436526 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c0485871114a0088ac6920bbbc3dbc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:26.438492 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:26.443366 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:26.490792 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c0485871114a0088ac6920bbbc3dbc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:26.491516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c0485871114a0088ac6920bbbc3dbc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:26.492138 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c0485871114a0088ac6920bbbc3dbc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:26.499097 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c0485871114a0088ac6920bbbc3dbc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:26.506103 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c0485871114a0088ac6920bbbc3dbc'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:26.523724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1432/2849] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:30:26 2022] DELETE /identity/v3/users/12c0485871114a0088ac6920bbbc3dbc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:26.528257 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:26.528767 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:26.528873 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:26.548063 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:26.549264 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:26.609684 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1418/2850] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:26.620358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:26.622414 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:26.679720 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ea5d9-cfdf-4f5f-9847-8c7698549a9a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:26.683218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:26.683891 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:26.684174 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:26.684446 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:26.693380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:26.695391 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:26.704671 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1433/2851] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:26.981457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:26.983145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:26.996182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53767242-3d63-48c1-bc31-f8240811f587 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:27.001284 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:27.002196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:27.002484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:27.002772 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] PATH_INFO: `/v3/projects/ac679b3abab440f3a9adf0bffba6a7b3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:27.004006 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac679b3abab440f3a9adf0bffba6a7b3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:27.006196 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:27.023864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:27.055368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac679b3abab440f3a9adf0bffba6a7b3', 'request_id': 'req-8ba1978f-867d-4dae-b24c-640aa88e41f4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.062944 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac679b3abab440f3a9adf0bffba6a7b3', 'request_id': 'req-8ba1978f-867d-4dae-b24c-640aa88e41f4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.070426 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac679b3abab440f3a9adf0bffba6a7b3', 'request_id': 'req-8ba1978f-867d-4dae-b24c-640aa88e41f4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.080917 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] The token cache is being invalidate because project ac679b3abab440f3a9adf0bffba6a7b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:30:27.081204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.081866 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.083436 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1419/2852] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:30:26 2022] DELETE /identity/v3/projects/ac679b3abab440f3a9adf0bffba6a7b3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:27.087915 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:27.097390 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43e238-4987-4884-abb1-b43ffb91cf3c service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:27.157763 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:27.158645 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:27.158932 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:27.159214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] PATH_INFO: `/v3/users/5fb5ffdb4bc24e04b3976fd3b0612288` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:27.160401 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb5ffdb4bc24e04b3976fd3b0612288)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:27.162429 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:27.191988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:27.211749 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb5ffdb4bc24e04b3976fd3b0612288'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.212388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb5ffdb4bc24e04b3976fd3b0612288'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.212981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb5ffdb4bc24e04b3976fd3b0612288'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.219586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb5ffdb4bc24e04b3976fd3b0612288'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.226926 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb5ffdb4bc24e04b3976fd3b0612288'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.244484 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1434/2853] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:30:27 2022] DELETE /identity/v3/users/5fb5ffdb4bc24e04b3976fd3b0612288 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:27.515015 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:27.516522 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:27.533947 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1978f-867d-4dae-b24c-640aa88e41f4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:27.592940 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:27.593837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:27.594123 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:27.594399 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] PATH_INFO: `/v3/projects/644d9f4c8f744ff2aeb300795c0d65ce` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:27.595581 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644d9f4c8f744ff2aeb300795c0d65ce)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:27.597545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:27.650547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644d9f4c8f744ff2aeb300795c0d65ce', 'request_id': 'req-27a7c348-4065-480d-941b-080752de0b58'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.657778 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644d9f4c8f744ff2aeb300795c0d65ce', 'request_id': 'req-27a7c348-4065-480d-941b-080752de0b58'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.664744 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644d9f4c8f744ff2aeb300795c0d65ce', 'request_id': 'req-27a7c348-4065-480d-941b-080752de0b58'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.676294 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] The token cache is being invalidate because project 644d9f4c8f744ff2aeb300795c0d65ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:30:27.676580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.677214 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.679453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1420/2854] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:30:27 2022] DELETE /identity/v3/projects/644d9f4c8f744ff2aeb300795c0d65ce => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:27.683440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:27.684883 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:27.698342 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ab14f-fe4b-49ea-8cd5-5aa7d44f8a39 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:27.755037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:27.756022 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:27.756308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:27.757516 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] PATH_INFO: `/v3/users/642cbd61016947ffa25043b214205269` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:27.759050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=642cbd61016947ffa25043b214205269)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:27.761487 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:27.817069 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642cbd61016947ffa25043b214205269'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.818072 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642cbd61016947ffa25043b214205269'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.818685 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642cbd61016947ffa25043b214205269'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.833338 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642cbd61016947ffa25043b214205269'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.844537 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1e91b72a-b8bb-4409-b325-74dff23757cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642cbd61016947ffa25043b214205269'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:27.865816 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1435/2855] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:30:27 2022] DELETE /identity/v3/users/642cbd61016947ffa25043b214205269 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:28.183110 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:28.184657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:28.201106 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7c348-4065-480d-941b-080752de0b58 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:28.237285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:28.269335 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:28.270250 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.270542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.270825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] PATH_INFO: `/v3/projects/235c531ded114f1cbb588408813884a0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.272029 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=235c531ded114f1cbb588408813884a0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:28.274172 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:28.328677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235c531ded114f1cbb588408813884a0', 'request_id': 'req-5d8de182-3349-4bbd-acdc-4216f0b576e4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:28.336067 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235c531ded114f1cbb588408813884a0', 'request_id': 'req-5d8de182-3349-4bbd-acdc-4216f0b576e4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:28.343853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235c531ded114f1cbb588408813884a0', 'request_id': 'req-5d8de182-3349-4bbd-acdc-4216f0b576e4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:28.356158 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] The token cache is being invalidate because project 235c531ded114f1cbb588408813884a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:30:28.356483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:28.357118 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5d8de182-3349-4bbd-acdc-4216f0b576e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:28.358708 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1421/2856] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:30:28 2022] DELETE /identity/v3/projects/235c531ded114f1cbb588408813884a0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:28.363826 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99f4e3-c8b7-4ce1-91eb-8265a122a746 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.364186 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99f4e3-c8b7-4ce1-91eb-8265a122a746 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.364505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99f4e3-c8b7-4ce1-91eb-8265a122a746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.386059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9f99f4e3-c8b7-4ce1-91eb-8265a122a746 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:28.387481 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f99f4e3-c8b7-4ce1-91eb-8265a122a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:28.409092 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:28.458660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1436/2857] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:28.520361 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932e40-aaee-42d9-ab65-4454262e1ac7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.520765 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932e40-aaee-42d9-ab65-4454262e1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.520937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932e40-aaee-42d9-ab65-4454262e1ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.542032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cb932e40-aaee-42d9-ab65-4454262e1ac7 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:28.543233 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cb932e40-aaee-42d9-ab65-4454262e1ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:28.553068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1422/2858] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:28.558582 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99f4e3-c8b7-4ce1-91eb-8265a122a746 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:28.569011 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99f4e3-c8b7-4ce1-91eb-8265a122a746 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:28.573586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:28.574291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.574575 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.574852 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.575616 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:28.575956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:28.577567 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:28.592857 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1437/2859] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:28.597278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932e40-aaee-42d9-ab65-4454262e1ac7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:28.608338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932e40-aaee-42d9-ab65-4454262e1ac7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:28.614182 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:28.614570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.614853 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.615131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.615821 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:28.617962 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:28.646127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1423/2860] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:30:28 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:28.651515 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:28.662990 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb75bc-1168-42b1-b158-e044fde6fb27 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:28.723674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:28.724650 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.724933 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.725210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.726060 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:28.728035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:28.751823 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1438/2861] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:30:28 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:30:28.755739 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:28.767159 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1edd0-339a-4acd-81a9-841edb0a8ab3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:28.772595 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:28.773268 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.773554 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.773867 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.774597 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:28.776197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:28.792579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1424/2862] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:30:28.796482 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:28.808465 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b368bbf-57bf-42b9-87d8-9c3f9153419b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:28.813269 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73412be-55f9-431b-b249-88f1618c83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:28.814096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73412be-55f9-431b-b249-88f1618c83bf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.814380 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73412be-55f9-431b-b249-88f1618c83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.814657 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73412be-55f9-431b-b249-88f1618c83bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.815374 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73412be-55f9-431b-b249-88f1618c83bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:28.817023 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73412be-55f9-431b-b249-88f1618c83bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:28.830596 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1439/2863] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:28.834888 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:28.845878 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa90c65-64f9-4d81-9727-7f9319f25bb9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:28.851028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:28.851932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.852216 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.852494 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] PATH_INFO: `/v3/projects/d4d90e448c9a4d46ba2f776e9afe0022/users/784de62741cb491b840e0180766f74f2/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.869632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d90e448c9a4d46ba2f776e9afe0022, user_id=784de62741cb491b840e0180766f74f2, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:28.872364 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:28.888557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1425/2864] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:28 2022] PUT /identity/v3/projects/d4d90e448c9a4d46ba2f776e9afe0022/users/784de62741cb491b840e0180766f74f2/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:28.899046 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059abcc-fb21-4692-ae01-6c3767847873 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:28.899330 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059abcc-fb21-4692-ae01-6c3767847873 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:28.899615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059abcc-fb21-4692-ae01-6c3767847873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:28.932768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6059abcc-fb21-4692-ae01-6c3767847873 None None] MFA Rules not processed for user `784de62741cb491b840e0180766f74f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:28.934147 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6059abcc-fb21-4692-ae01-6c3767847873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:29.003989 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1440/2865] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:29.015337 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:29.018649 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:29.093355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a662d2-27af-4e2e-928a-a1a41cceac8d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:29.097645 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3995f9-9f96-47f1-89f1-3d53d311e3e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:29.098162 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3995f9-9f96-47f1-89f1-3d53d311e3e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:29.098536 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3995f9-9f96-47f1-89f1-3d53d311e3e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:29.099258 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3995f9-9f96-47f1-89f1-3d53d311e3e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:29.110326 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3995f9-9f96-47f1-89f1-3d53d311e3e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:29.111728 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3995f9-9f96-47f1-89f1-3d53d311e3e2 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:29.122568 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1426/2866] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:39.000898 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:39.117591 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:39.308573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059abcc-fb21-4692-ae01-6c3767847873 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:39.370375 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059abcc-fb21-4692-ae01-6c3767847873 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:39.374938 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:39.375807 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:39.376094 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:39.376371 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] PATH_INFO: `/v3/users/784de62741cb491b840e0180766f74f2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:39.377615 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784de62741cb491b840e0180766f74f2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:39.379573 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:39.426985 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784de62741cb491b840e0180766f74f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:39.427643 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784de62741cb491b840e0180766f74f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:39.428759 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784de62741cb491b840e0180766f74f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:39.436280 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784de62741cb491b840e0180766f74f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:39.443274 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784de62741cb491b840e0180766f74f2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:39.461320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1441/2867] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:30:39 2022] DELETE /identity/v3/users/784de62741cb491b840e0180766f74f2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:39.466003 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:39.466301 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:39.466614 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:39.483706 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:39.484914 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:39.544653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1427/2868] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:39.554900 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:39.556311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:39.619696 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1bd5c-f739-4d40-90b0-3a1d3c678cf9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:39.623660 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c0555-97b6-4166-805c-6e35eae835d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:39.624320 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0555-97b6-4166-805c-6e35eae835d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:39.624626 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0555-97b6-4166-805c-6e35eae835d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:39.624991 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c0555-97b6-4166-805c-6e35eae835d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:39.635719 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c0555-97b6-4166-805c-6e35eae835d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:39.637308 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c0555-97b6-4166-805c-6e35eae835d0 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:39.647604 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1442/2869] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:39.936173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:39.937648 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:39.949615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557d447-06e6-441e-9a21-3786e1c7a813 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:39.954200 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:39.955059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:39.955502 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:39.956005 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] PATH_INFO: `/v3/projects/d4d90e448c9a4d46ba2f776e9afe0022` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:39.957285 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d90e448c9a4d46ba2f776e9afe0022)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:39.959432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.009664 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d90e448c9a4d46ba2f776e9afe0022', 'request_id': 'req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:40.017333 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d90e448c9a4d46ba2f776e9afe0022', 'request_id': 'req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:40.024908 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d90e448c9a4d46ba2f776e9afe0022', 'request_id': 'req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:40.036061 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] The token cache is being invalidate because project d4d90e448c9a4d46ba2f776e9afe0022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:30:40.036350 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:40.037007 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5e2f1e78-66b9-4d1d-9fb0-37efc7f3e521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:40.039176 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1428/2870] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:30:39 2022] DELETE /identity/v3/projects/d4d90e448c9a4d46ba2f776e9afe0022 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:40.044039 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6abb0-407e-4dbe-a564-9b4c8a406ef1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.044543 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6abb0-407e-4dbe-a564-9b4c8a406ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.044746 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6abb0-407e-4dbe-a564-9b4c8a406ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.062230 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-12a6abb0-407e-4dbe-a564-9b4c8a406ef1 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:40.063358 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-12a6abb0-407e-4dbe-a564-9b4c8a406ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:40.080648 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:40.126919 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1443/2871] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:40.135930 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7a220-538b-4d59-9d96-e6cf48896860 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.136511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7a220-538b-4d59-9d96-e6cf48896860 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.136511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7a220-538b-4d59-9d96-e6cf48896860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.155464 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-56d7a220-538b-4d59-9d96-e6cf48896860 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:40.156666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-56d7a220-538b-4d59-9d96-e6cf48896860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:40.165245 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1429/2872] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:40.169620 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6abb0-407e-4dbe-a564-9b4c8a406ef1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:40.179505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6abb0-407e-4dbe-a564-9b4c8a406ef1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:40.183958 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:40.184607 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.184890 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.185165 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.185914 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:40.186218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:40.187809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.200725 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1444/2873] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 16:30:40.205188 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7a220-538b-4d59-9d96-e6cf48896860 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:40.215679 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7a220-538b-4d59-9d96-e6cf48896860 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:40.220401 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:40.221069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.221355 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.221631 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.222358 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:40.224541 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.251481 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1430/2874] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:30:40 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:40.255674 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:40.266168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b6b49-b0fd-478c-9f5b-4ca4b7a14b4c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:40.324865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:40.324865 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.327137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.327137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.327137 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:40.328440 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.349490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1445/2875] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:30:40 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:40.353824 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:40.364032 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8968f-7b32-4497-a810-78d07c1ccc3b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:40.368683 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:40.369338 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.369623 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.369937 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.370653 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:40.372238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.383582 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1431/2876] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:40.387812 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:40.398707 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307260ba-eb84-4d75-9a21-21b28a960367 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:40.403266 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28717b1-7ed2-4ae7-b250-ae95f4c848c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:40.403916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28717b1-7ed2-4ae7-b250-ae95f4c848c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.404201 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28717b1-7ed2-4ae7-b250-ae95f4c848c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.404476 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28717b1-7ed2-4ae7-b250-ae95f4c848c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.405188 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28717b1-7ed2-4ae7-b250-ae95f4c848c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:40.406813 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28717b1-7ed2-4ae7-b250-ae95f4c848c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.416953 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1446/2877] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:40.421127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:40.431168 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c29c7-0103-43d1-9993-f470ebc5363b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:40.435837 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:40.436712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.436994 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.437272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] PATH_INFO: `/v3/projects/d98ce59f77974033afa3f8266ae7ee29/users/a97917d7426e46a899f141fd57641bea/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.452609 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d98ce59f77974033afa3f8266ae7ee29, user_id=a97917d7426e46a899f141fd57641bea, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:40.455309 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.470079 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1432/2878] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:40 2022] PUT /identity/v3/projects/d98ce59f77974033afa3f8266ae7ee29/users/a97917d7426e46a899f141fd57641bea/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:40.481096 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18f87b-c616-4a4f-9321-8bc0ed5b4974 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.481394 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18f87b-c616-4a4f-9321-8bc0ed5b4974 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.481789 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18f87b-c616-4a4f-9321-8bc0ed5b4974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.508388 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9f18f87b-c616-4a4f-9321-8bc0ed5b4974 None None] MFA Rules not processed for user `a97917d7426e46a899f141fd57641bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:40.509595 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f18f87b-c616-4a4f-9321-8bc0ed5b4974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:40.580397 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1447/2879] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:40.592310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:40.594064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:40.616606 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:40.654725 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ea81d-2f8e-4a2e-8a07-2998a2c20b20 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:40.658453 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b407a9-e1ed-4fb7-a793-4570bfcc4ae0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:40.659102 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b407a9-e1ed-4fb7-a793-4570bfcc4ae0 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:40.659388 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b407a9-e1ed-4fb7-a793-4570bfcc4ae0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:40.659668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b407a9-e1ed-4fb7-a793-4570bfcc4ae0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:40.668345 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b407a9-e1ed-4fb7-a793-4570bfcc4ae0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:40.669804 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b407a9-e1ed-4fb7-a793-4570bfcc4ae0 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:40.679169 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1433/2880] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:48.816059 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18f87b-c616-4a4f-9321-8bc0ed5b4974 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:48.836498 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:48.887667 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18f87b-c616-4a4f-9321-8bc0ed5b4974 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:48.892623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:48.893533 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.893908 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.894089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] PATH_INFO: `/v3/users/a97917d7426e46a899f141fd57641bea` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:48.895463 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a97917d7426e46a899f141fd57641bea)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:48.898523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:48.954283 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a97917d7426e46a899f141fd57641bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:48.955081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a97917d7426e46a899f141fd57641bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:48.955726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a97917d7426e46a899f141fd57641bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:48.964665 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a97917d7426e46a899f141fd57641bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:48.972085 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a97917d7426e46a899f141fd57641bea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:48.992040 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1448/2881] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:30:48 2022] DELETE /identity/v3/users/a97917d7426e46a899f141fd57641bea => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:48.996991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:48.997539 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:48.997662 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:49.015449 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:49.016668 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:49.089319 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1434/2882] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:49.101368 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:49.104024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:49.171425 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4aab66-d2d2-4bba-ad12-74b47e24acbc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:49.175089 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e11892-d633-486b-89ea-69c80386cb1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:49.175743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e11892-d633-486b-89ea-69c80386cb1a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:49.176028 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e11892-d633-486b-89ea-69c80386cb1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:49.176305 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e11892-d633-486b-89ea-69c80386cb1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:49.186676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e11892-d633-486b-89ea-69c80386cb1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:49.187981 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e11892-d633-486b-89ea-69c80386cb1a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:49.199772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1449/2883] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:49.782640 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:49.784680 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:49.800792 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c86385-ca77-4c14-ac3c-a816aae6b403 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:49.809679 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:49.811742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:49.812081 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:49.812395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] PATH_INFO: `/v3/projects/d98ce59f77974033afa3f8266ae7ee29` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:49.813932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d98ce59f77974033afa3f8266ae7ee29)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:49.816149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:49.875163 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd98ce59f77974033afa3f8266ae7ee29', 'request_id': 'req-b836b817-9e57-4b37-b949-279b791e2017'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:49.885511 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd98ce59f77974033afa3f8266ae7ee29', 'request_id': 'req-b836b817-9e57-4b37-b949-279b791e2017'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:49.894117 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd98ce59f77974033afa3f8266ae7ee29', 'request_id': 'req-b836b817-9e57-4b37-b949-279b791e2017'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:49.907395 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] The token cache is being invalidate because project d98ce59f77974033afa3f8266ae7ee29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:30:49.907716 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:49.908374 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b836b817-9e57-4b37-b949-279b791e2017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:30:49.910068 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1435/2884] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:30:49 2022] DELETE /identity/v3/projects/d98ce59f77974033afa3f8266ae7ee29 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:49.915349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75a219-d333-45f6-8c03-6f67befa1ec2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:49.916177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75a219-d333-45f6-8c03-6f67befa1ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:49.916177 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75a219-d333-45f6-8c03-6f67befa1ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:49.936743 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9b75a219-d333-45f6-8c03-6f67befa1ec2 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:49.938030 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9b75a219-d333-45f6-8c03-6f67befa1ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:49.977370 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:50.015988 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1450/2885] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:50.025607 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9dc78-14a4-4c1a-8de8-9d73332c4318 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.026439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9dc78-14a4-4c1a-8de8-9d73332c4318 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.026439 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9dc78-14a4-4c1a-8de8-9d73332c4318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.046190 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ffd9dc78-14a4-4c1a-8de8-9d73332c4318 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:50.047415 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ffd9dc78-14a4-4c1a-8de8-9d73332c4318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:50.056414 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1436/2886] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:50.060830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75a219-d333-45f6-8c03-6f67befa1ec2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:50.074175 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75a219-d333-45f6-8c03-6f67befa1ec2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:50.079037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:50.079721 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.080007 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.080286 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.081024 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 16:30:50.081332 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:50.082999 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:50.102814 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1451/2887] 10.222.0.88 () {58 vars in 1152 bytes} [Thu Jun 16 16:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 16:30:50.110530 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9dc78-14a4-4c1a-8de8-9d73332c4318 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:50.123310 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9dc78-14a4-4c1a-8de8-9d73332c4318 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:50.129457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:50.130195 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.131238 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.131549 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.132456 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:50.135177 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:50.164721 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1437/2888] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:30:50 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:30:50.169326 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:50.180977 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f5b29f-e2d6-4380-aba6-eb7826bd441c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:50.267850 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:50.269021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.269525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.270045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.271227 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:50.274701 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:50.298522 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1452/2889] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:30:50 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:50.302651 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:50.313709 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbaf14-95b4-4279-b78f-eaceb45fc8ec admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:50.318711 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:50.319497 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.319783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.320064 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.321180 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:50.322904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:50.335383 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1438/2890] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:50.343698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:50.355029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f782e-dc61-4cc5-952b-721706e41c86 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:50.359877 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73f945-faa2-4809-b629-01ad31ef1250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:50.360555 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73f945-faa2-4809-b629-01ad31ef1250 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.360845 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73f945-faa2-4809-b629-01ad31ef1250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.361128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73f945-faa2-4809-b629-01ad31ef1250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.361995 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73f945-faa2-4809-b629-01ad31ef1250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:50.363525 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73f945-faa2-4809-b629-01ad31ef1250 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:50.375035 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1453/2891] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:50.380271 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:50.391713 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaba32d-b64b-45b5-879d-286311c7f504 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:50.398496 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:50.399824 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.400274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.400632 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] PATH_INFO: `/v3/projects/3643cc8acf58444383e525f1990d1129/users/5bd17610e5f44d70a9f3f83abfc1e6c8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.403366 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:50.425483 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3643cc8acf58444383e525f1990d1129, user_id=5bd17610e5f44d70a9f3f83abfc1e6c8, role_id=d3e1bf46ae9d4e86a9480cd0a49ee5e2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:50.428228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:50.448712 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1439/2892] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:50 2022] PUT /identity/v3/projects/3643cc8acf58444383e525f1990d1129/users/5bd17610e5f44d70a9f3f83abfc1e6c8/roles/d3e1bf46ae9d4e86a9480cd0a49ee5e2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:30:50.455485 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77619d2-4670-4cf8-9eae-3b7b10ab2f67 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.455855 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77619d2-4670-4cf8-9eae-3b7b10ab2f67 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.456311 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77619d2-4670-4cf8-9eae-3b7b10ab2f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.483687 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c77619d2-4670-4cf8-9eae-3b7b10ab2f67 None None] MFA Rules not processed for user `fded449262ae4659b5d91d4b0bc2b7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:50.485655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c77619d2-4670-4cf8-9eae-3b7b10ab2f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:50.554081 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1454/2893] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:50.564251 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:50.566109 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:50.629445 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7669177-d6e4-42c3-9981-faeeeddc2a96 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:50.634479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:50.635183 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:50.635506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:50.635786 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:50.647479 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:50.649037 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:50.661617 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1440/2894] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:30:55.411638 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7cc75-85f5-442f-81fc-173e77c76eda None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:55.412254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7cc75-85f5-442f-81fc-173e77c76eda None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:55.412254 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7cc75-85f5-442f-81fc-173e77c76eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:55.445825 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51e7cc75-85f5-442f-81fc-173e77c76eda None None] MFA Rules not processed for user `5bd17610e5f44d70a9f3f83abfc1e6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:30:55.447068 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51e7cc75-85f5-442f-81fc-173e77c76eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:30:55.496207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:30:55.538166 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1455/2895] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:30:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:30:55.543069 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:55.555059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacfb38-0ed7-488a-bee7-f220c20f3730 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:55.560375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:55.561053 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:55.561375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:55.562634 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:55.563367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:55.565334 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:55.608385 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1441/2896] 10.222.0.88 () {60 vars in 1152 bytes} [Thu Jun 16 16:30:55 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 16:30:55.612427 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7cc75-85f5-442f-81fc-173e77c76eda None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:55.624075 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7cc75-85f5-442f-81fc-173e77c76eda None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:55.695029 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:55.696450 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:55.696909 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:55.697780 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:55.698996 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:55.700618 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:55.721317 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1456/2897] 10.222.0.88 () {60 vars in 1143 bytes} [Thu Jun 16 16:30:55 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 16:30:55.726542 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:55.738624 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97159dfb-3ec2-4706-9fca-ac721917f1d3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:55.744031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:55.744704 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:55.744991 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:55.745267 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:55.746020 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:55.747629 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:55.759543 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1442/2898] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 16:30:55.763748 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:55.774392 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee4551-fad1-4a2a-9b39-1fb0deac0410 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:55.779112 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:55.780019 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:55.780307 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:55.780586 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] PATH_INFO: `/v3/projects/d5d2a6d7d86d47ca9230c9bc3872ab77/users/84ba7710923442b6b49968bad82447e0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:55.796080 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d2a6d7d86d47ca9230c9bc3872ab77, user_id=84ba7710923442b6b49968bad82447e0, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:55.798810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:55.815634 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1457/2899] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:55 2022] PUT /identity/v3/projects/d5d2a6d7d86d47ca9230c9bc3872ab77/users/84ba7710923442b6b49968bad82447e0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:30:55.820059 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:55.830367 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc63526-0eac-432d-8866-3c9337113e29 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:55.891932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524a3c9-040e-4c07-8113-e28c5ab7411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:55.892921 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a3c9-040e-4c07-8113-e28c5ab7411f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:55.893204 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a3c9-040e-4c07-8113-e28c5ab7411f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:55.893574 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a3c9-040e-4c07-8113-e28c5ab7411f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:55.894240 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524a3c9-040e-4c07-8113-e28c5ab7411f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:55.895868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524a3c9-040e-4c07-8113-e28c5ab7411f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:55.907290 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1443/2900] 10.222.0.88 () {58 vars in 1121 bytes} [Thu Jun 16 16:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 16:30:55.911907 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:30:55.923686 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d663d00-8c46-4ed7-ba0d-56d1d4b4039f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:30:55.929045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:30:55.930027 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:30:55.930319 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:30:55.930597 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] PATH_INFO: `/v3/projects/d5d2a6d7d86d47ca9230c9bc3872ab77/users/84ba7710923442b6b49968bad82447e0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:30:55.932568 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5d2a6d7d86d47ca9230c9bc3872ab77, user_id=84ba7710923442b6b49968bad82447e0, role_id=d3e29c31885f4a7da9a2d5bf83cd3d79)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:30:55.935295 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:30:55.947694 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1458/2901] 10.222.0.88 () {60 vars in 1482 bytes} [Thu Jun 16 16:30:55 2022] PUT /identity/v3/projects/d5d2a6d7d86d47ca9230c9bc3872ab77/users/84ba7710923442b6b49968bad82447e0/roles/d3e29c31885f4a7da9a2d5bf83cd3d79 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:31:00.648022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ed157-611c-4ba3-9651-60d0cb7a60ef None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:00.648601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ed157-611c-4ba3-9651-60d0cb7a60ef None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:00.648601 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ed157-611c-4ba3-9651-60d0cb7a60ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:00.683559 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-297ed157-611c-4ba3-9651-60d0cb7a60ef None None] MFA Rules not processed for user `84ba7710923442b6b49968bad82447e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:31:00.684825 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-297ed157-611c-4ba3-9651-60d0cb7a60ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:00.734548 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:00.773557 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1444/2902] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:31:00.790787 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:00.794673 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:00.881885 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81949f07-03c6-4ca7-9139-b1b5e8d5f85c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:00.888327 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:00.888927 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:00.889210 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:00.889490 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:00.916530 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:00.956122 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:00.957791 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:00.968037 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1459/2903] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:31:10.769844 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=12 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:10.963994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=16 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:27.052946 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:27.053547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:27.053547 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:27.071916 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] MFA Rules not processed for user `5bd17610e5f44d70a9f3f83abfc1e6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:31:27.073107 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:27.080028 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1445/2904] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:31:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:31:27.398605 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:27.400033 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:27.457130 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2296c-12b5-4f00-b5c6-f598aab5439a service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:27.461212 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:27.461893 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:27.462200 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:27.462464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:27.464218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:27.473828 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:27.476168 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:27.484172 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1460/2905] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:31:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:31:27.920799 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:27.922210 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:28.012567 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7604ce4-331c-4795-8348-e7219eecc785 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:28.017504 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:28.018186 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:28.018473 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:28.018751 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:28.020525 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:28.031868 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:28.033477 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:28.043519 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1446/2906] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:31:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:31:37.480644 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=26 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:38.039191 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=35 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:41.650600 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:41.652765 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:41.670523 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6918e6-ae65-4463-a03f-f3783404cf4f service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:41.676018 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:41.676672 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:41.676956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:41.677918 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:41.680109 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:41.694049 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:41.695655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:41.707076 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1461/2907] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:31:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:31:44.623127 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:44.634835 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae06b6f-8fa1-4327-956f-56b244bfdf32 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:44.638612 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:44.639292 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:44.639580 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:44.640022 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:44.650278 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:44.651730 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:44.664446 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1447/2908] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:31:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:31:46.318291 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:46.330231 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf3340-3deb-43e1-8f1a-ea622d7095db service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:46.336726 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:46.337393 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:46.337681 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:46.337994 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:46.339837 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:46.354339 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:46.356002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:46.368949 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1462/2909] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:31:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:31:46.470418 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:31:46.486173 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b738b67-b234-47c9-8b3d-60b74593ac6e service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:31:46.493131 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:31:46.493859 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:31:46.494784 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:31:46.495312 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:31:46.497209 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:31:46.510689 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:31:46.512382 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:31:46.523615 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1448/2910] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:31:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:31:56.363714 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:31:56.518740 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=12 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:33:45.156218 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:33:45.166486 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa9e1a-c789-442c-991c-239077553375 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:33:45.170110 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:33:45.170760 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:33:45.171045 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:33:45.171313 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:33:45.173101 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:33:45.183215 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:33:45.184752 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:33:45.194102 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1463/2911] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:33:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:33:55.190318 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:34:14.371972 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:14.382145 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac0acd-9346-40ef-aa8c-817c25f55cba service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:14.386012 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9613b5-a011-404d-84b1-c2ce9df9dd9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:14.386691 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9613b5-a011-404d-84b1-c2ce9df9dd9c service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:14.387031 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9613b5-a011-404d-84b1-c2ce9df9dd9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:14.387524 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9613b5-a011-404d-84b1-c2ce9df9dd9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:14.389421 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cf9613b5-a011-404d-84b1-c2ce9df9dd9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:14.404457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9613b5-a011-404d-84b1-c2ce9df9dd9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:14.405879 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9613b5-a011-404d-84b1-c2ce9df9dd9c service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:14.418982 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1449/2912] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:15.611800 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:15.620882 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d04fac-8d85-469d-80e3-86ae6d2df321 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:15.624987 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:15.625641 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:15.625956 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:15.626246 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:15.628090 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:15.642527 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:15.644155 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:15.655892 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1464/2913] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:19.099506 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:19.100043 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:19.100147 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:19.118493 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] MFA Rules not processed for user `5bd17610e5f44d70a9f3f83abfc1e6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:34:19.119702 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:19.126737 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1450/2914] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:34:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:34:19.458833 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:34:19.468830 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470b2a-8ab8-4377-89e9-38875922f449 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:34:19.472964 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:34:19.473614 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:34:19.473931 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:34:19.474214 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:34:19.476043 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:34:19.488271 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:34:19.489829 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:34:19.498676 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1465/2915] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:34:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:34:29.112820 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=9 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:34:29.494864 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:36:39.241758 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:39.243143 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:39.369040 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a617d-1ef0-423b-80eb-f0c016e728b6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:39.372860 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c120f-4536-4830-bff4-885f800cee3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '984fa8bcb66146e5ae376a16d1d1b3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:39.373515 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c120f-4536-4830-bff4-885f800cee3a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:39.373832 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c120f-4536-4830-bff4-885f800cee3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:39.374657 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c120f-4536-4830-bff4-885f800cee3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:39.399910 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c120f-4536-4830-bff4-885f800cee3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:39.401261 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c120f-4536-4830-bff4-885f800cee3a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:39.410637 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1451/2916] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:36:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:36:49.407622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=54 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:36:50.450000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:50.501294 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c75661-fa1c-432d-8dc8-a758f4722a2b service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:50.508184 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:50.508851 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:50.509139 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:50.509411 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:50.519125 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:50.520628 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:50.532512 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1466/2917] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:36:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:36:51.488352 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2f595-612f-42f8-9dda-be86d09bb722 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:51.489281 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2f595-612f-42f8-9dda-be86d09bb722 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:51.489579 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2f595-612f-42f8-9dda-be86d09bb722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:51.508076 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-24a2f595-612f-42f8-9dda-be86d09bb722 None None] MFA Rules not processed for user `5bd17610e5f44d70a9f3f83abfc1e6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 16:36:51.509306 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-24a2f595-612f-42f8-9dda-be86d09bb722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:51.517146 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1452/2918] 10.222.0.88 () {58 vars in 957 bytes} [Thu Jun 16 16:36:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 16:36:52.299207 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:52.300625 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:52.329428 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430ee7b-2959-42c9-ab54-5abd8a4ec5ef service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:52.334870 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:52.335532 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:52.335818 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:52.336136 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:52.346409 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:52.348410 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:52.359700 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1467/2919] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:36:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:36:55.169490 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2f595-612f-42f8-9dda-be86d09bb722 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:36:55.210666 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2f595-612f-42f8-9dda-be86d09bb722 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:36:55.216791 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:36:55.217904 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:36:55.218373 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:36:55.218826 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:36:55.221088 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:36:55.252269 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:36:55.254026 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:36:55.272771 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1453/2920] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:36:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 16:37:02.354966 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:37:05.205321 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:37:05.221647 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783715b9-ab8d-4642-b828-7a56c97e38c6 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:37:05.227924 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee056b132bd24851bd0e2afe9cb7170e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:37:05.229000 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:37:05.229284 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:37:05.229583 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:37:05.240050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:37:05.241656 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:37:05.254416 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1468/2921] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:37:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:37:05.262149 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=21 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:37:08.039095 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:37:08.060545 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb15a7-3ebe-4ba1-89ba-97ea038d8b31 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:37:08.064829 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:37:08.065478 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:37:08.065783 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:37:08.066066 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:37:08.075983 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:37:08.077432 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:37:08.087961 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1454/2922] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:37:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:37:15.248198 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:37:18.083082 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=10 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:38:53.147182 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:38:53.158768 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a0c3b-0294-422f-a8c4-dbcd0554f90a service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:38:53.162384 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:38:53.163065 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:38:53.163349 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:38:53.163655 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:38:53.171588 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:38:53.173020 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:38:53.180965 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1469/2923] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:38:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:39:03.177333 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:39:14.799167 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:39:14.808228 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549d72-5284-4c77-86c8-fdf5ef85b5d4 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:39:14.812165 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33e61ed31e0943e8baf9a85412860245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:39:14.812803 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:39:14.813092 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:39:14.813375 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:39:14.884046 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:39:14.885484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:39:14.903932 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1455/2924] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:39:18.054504 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:39:18.067190 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f07ea-05d4-40bf-8759-2968f1bee8c1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:39:18.072298 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:39:18.072975 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:39:18.073261 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:39:18.073541 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:39:18.075556 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:39:18.089441 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:39:18.091464 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:39:18.105688 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1470/2925] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:39:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:39:20.920424 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:39:20.943675 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e8b3e-86f8-415d-90ab-50dce9b59ef2 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:39:20.951055 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:39:20.951979 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:39:20.952694 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:39:20.955197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] PATH_INFO: `/v3/users/5bd17610e5f44d70a9f3f83abfc1e6c8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:39:20.955197 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd17610e5f44d70a9f3f83abfc1e6c8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:39:20.956394 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:39:21.004093 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd17610e5f44d70a9f3f83abfc1e6c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.004742 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd17610e5f44d70a9f3f83abfc1e6c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.011677 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd17610e5f44d70a9f3f83abfc1e6c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.012324 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd17610e5f44d70a9f3f83abfc1e6c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.019274 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd17610e5f44d70a9f3f83abfc1e6c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.037080 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1456/2926] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:39:20 2022] DELETE /identity/v3/users/5bd17610e5f44d70a9f3f83abfc1e6c8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:39:21.217157 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:39:21.235161 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131f7e-654a-41d0-877d-11c5fe479db2 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:39:21.291916 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:39:21.292771 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:39:21.293050 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:39:21.293323 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] PATH_INFO: `/v3/projects/3643cc8acf58444383e525f1990d1129` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:39:21.294505 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3643cc8acf58444383e525f1990d1129)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:39:21.296426 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:39:21.341862 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3643cc8acf58444383e525f1990d1129', 'request_id': 'req-4a60a740-3d06-43db-83cc-6b4f9ad13220'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.349622 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3643cc8acf58444383e525f1990d1129', 'request_id': 'req-4a60a740-3d06-43db-83cc-6b4f9ad13220'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.356021 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3643cc8acf58444383e525f1990d1129', 'request_id': 'req-4a60a740-3d06-43db-83cc-6b4f9ad13220'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.377848 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] The token cache is being invalidate because project 3643cc8acf58444383e525f1990d1129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:39:21.378141 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.378772 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.380310 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1471/2927] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:39:21 2022] DELETE /identity/v3/projects/3643cc8acf58444383e525f1990d1129 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:39:21.384272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:39:21.385789 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:39:21.399622 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6a743-5fd2-4c51-b1f2-abd2928d547f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:39:21.446864 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:39:21.460886 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:39:21.461628 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:39:21.461949 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:39:21.462254 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] PATH_INFO: `/v3/users/84ba7710923442b6b49968bad82447e0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:39:21.463457 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ba7710923442b6b49968bad82447e0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:39:21.465368 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:39:21.514136 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ba7710923442b6b49968bad82447e0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.515484 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ba7710923442b6b49968bad82447e0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.522570 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ba7710923442b6b49968bad82447e0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.523201 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ba7710923442b6b49968bad82447e0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.529811 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ba7710923442b6b49968bad82447e0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.547272 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1457/2928] 10.222.0.88 () {58 vars in 1223 bytes} [Thu Jun 16 16:39:21 2022] DELETE /identity/v3/users/84ba7710923442b6b49968bad82447e0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 16:39:21.717691 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:39:21.719150 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:39:21.732208 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60a740-3d06-43db-83cc-6b4f9ad13220 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:39:21.790296 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fded449262ae4659b5d91d4b0bc2b7a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a913cabdc8c6449f971e776cd4ba8813', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:39:21.791146 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:39:21.791425 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:39:21.791698 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] PATH_INFO: `/v3/projects/d5d2a6d7d86d47ca9230c9bc3872ab77` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:39:21.792872 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5d2a6d7d86d47ca9230c9bc3872ab77)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:39:21.794810 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:39:21.802383 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:39:21.841809 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5d2a6d7d86d47ca9230c9bc3872ab77', 'request_id': 'req-621ed7ca-2d0e-42d9-849e-1a66a570ca85'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.849790 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5d2a6d7d86d47ca9230c9bc3872ab77', 'request_id': 'req-621ed7ca-2d0e-42d9-849e-1a66a570ca85'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.856156 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5d2a6d7d86d47ca9230c9bc3872ab77', 'request_id': 'req-621ed7ca-2d0e-42d9-849e-1a66a570ca85'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.866724 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] The token cache is being invalidate because project d5d2a6d7d86d47ca9230c9bc3872ab77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 16:39:21.867002 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.867623 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-621ed7ca-2d0e-42d9-849e-1a66a570ca85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 16:39:21.869128 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1472/2929] 10.222.0.88 () {58 vars in 1232 bytes} [Thu Jun 16 16:39:21 2022] DELETE /identity/v3/projects/d5d2a6d7d86d47ca9230c9bc3872ab77 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 16:39:31.543818 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:39:31.863762 dv-cinder-837354-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 16:41:58.477379 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 16:41:58.478848 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.549827 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c31d-6e6a-4af7-a515-3fa15709496c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 16:41:58.607480 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb02a39-933e-44b4-a412-8d2f27a15e37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7512a42da9154c008dad057c5508a02b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af8309ed84b44895b09b00c0bb184778', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 16:41:58.608125 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02a39-933e-44b4-a412-8d2f27a15e37 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 16:41:58.608411 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02a39-933e-44b4-a412-8d2f27a15e37 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 16:41:58.608687 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb02a39-933e-44b4-a412-8d2f27a15e37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 16:41:58.610492 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eeb02a39-933e-44b4-a412-8d2f27a15e37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 16:41:58.622220 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb02a39-933e-44b4-a412-8d2f27a15e37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 16:41:58.623802 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb02a39-933e-44b4-a412-8d2f27a15e37 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 16:41:58.634224 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1458/2930] 10.222.0.88 () {60 vars in 1365 bytes} [Thu Jun 16 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 16:42:08.629942 dv-cinder-837354-2-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=58 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}